Augmented reality bindings of physical objects and virtual objects

ABSTRACT

Systems, apparatus, and computer-readable media for managing data storage for generating virtual bindings are provided. In embodiments, a user may perform one or more gestures and/or voice commands to create virtual bindings with physical objects, where the created virtual bindings may take on attributes and create/perform actions based on attributes of the physical objects. A projection device may recognize the physical objects and cause the bindings and/or projected virtual objects to perform various actions in response to different user gestures and/or voice commands. Additionally, the system may instruct some physical objects (e.g., robots, electromechanical devices, etc.) in response to user gestures/voice commands to cause those physical devices to perform various actions. Other embodiments are described and/or claimed.

TECHNICAL FIELD

The present disclosure relates to the field of augmented reality systemsand devices, and in particular, to apparatuses, methods, and storagemedia for creating and managing virtual bindings between augmentedreality objects and physical objects.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Unless otherwiseindicated herein, the materials described in this section are not priorart to the claims in this application and are not admitted to be priorart by inclusion in this section.

Augmented reality (AR) systems include devices that augment a user'sview of a physical (i.e., “real-world”) environment withcomputer-generated sensory inputs, such as sound, video, graphics,haptic feedback, etc. Existing AR systems do not provide mechanisms thatattach or associate physical objects with the computer-generated sensoryinputs.

BRIEF DESCRIPTION OF THE DRAWINGS

The included drawings are for illustrative purposes and serve to provideexamples of possible structures and operations for the disclosedembodiments. The drawings in no way limit any changes in form and detailthat may be made by one skilled in the art without departing from thespirit and scope of the disclosed concepts.

FIG. 1 illustrates an environment in which various embodiments may bepracticed.

FIG. 2 illustrates an example implementation of a computer device, inaccordance with various embodiments.

FIG. 3 illustrates an example of non-transitory computer-readablestorage media that may be suitable for use to store instructions thatcause an apparatus, in response to execution of the instructions by theapparatus, to practice selected aspects of the present disclosure

FIG. 4 illustrates an example process in accordance with variousembodiments.

FIG. 5 illustrates another example process in accordance with variousembodiments.

FIGS. 6-7 illustrate an example of user interactions with physicalobjects in accordance with various embodiments.

FIG. 8 illustrates an example of user interactions with instrumentedobjects in accordance with various embodiments.

DETAILED DESCRIPTION

Disclosed embodiments are related to user interaction devices forinteracting with augmented reality, and in particular, technologies thatallow a user to interact with virtual objects in a user domain (i.e.,“real-world”). Currently available augmented reality systems may allowusers to interact with virtual objects displayed on a display device(e.g., televisions, tablet computers, etc.), where the user may interactwith the virtual objects by performing various gestures that arecaptured by a motion capture device. In embodiments, a user may performone or more gesture to create virtual bindings with real (physical)objects, where the created virtual bindings may take on attributes andcreate/perform actions based on the real objects. A projection devicemay recognize the real (physical) objects and cause the bindings and/orprojected virtual objects to perform various actions in response todifferent user gestures and/or voice commands. Additionally, the systemmay send instructions to instrumented physical objects (e.g., robots,electromechanical devices, etc.) in response to user gestures/voicecommands to cause those physical devices to perform various actions.

The following detailed description refers to the accompanying drawings.The same reference numbers may be used in different drawings to identifythe same or similar elements. In the following description, for purposesof explanation and not limitation, specific details are set forth suchas particular structures, architectures, interfaces, techniques, etc.,in order to provide a thorough understanding of the various aspects ofthe claimed invention. However, it will be apparent to those skilled inthe art having the benefit of the present disclosure that the variousaspects of the invention claimed may be practiced in other examples thatdepart from these specific details. In certain instances, descriptionsof well-known devices, circuits, and methods are omitted so as not toobscure the description of the present invention with unnecessarydetail.

Various aspects of the illustrative embodiments will be described usingterms commonly employed by those skilled in the art to convey thesubstance of their work to others skilled in the art. However, it willbe apparent to those skilled in the art that alternate embodiments maybe practiced with only some of the described aspects. For purposes ofexplanation, specific numbers, materials, and configurations are setforth in order to provide a thorough understanding of the illustrativeembodiments. However, it will be apparent to one skilled in the art thatalternate embodiments may be practiced without the specific details. Inother instances, well-known features are omitted or simplified in ordernot to obscure the illustrative embodiments.

Further, various operations will be described as multiple discreteoperations, in turn, in a manner that is most helpful in understandingthe illustrative embodiments; however, the order of description shouldnot be construed as to imply that these operations are necessarily orderdependent. In particular, these operations need not be performed in theorder of presentation.

The phrase “in various embodiments,” “in some embodiments,” and the likeare used repeatedly. The phrase generally does not refer to the sameembodiments; however, it may. The terms “comprising,” “having,” and“including” are synonymous, unless the context dictates otherwise. Thephrase “A and/or B” means (A), (B), or (A and B). The phrases “A/B” and“A or B” mean (A), (B), or (A and B), similar to the phrase “A and/orB.” For the purposes of the present disclosure, the phrase “at least oneof A and B” means (A), (B), or (A and B). The description may use thephrases “in an embodiment,” “in embodiments,” “in some embodiments,”and/or “in various embodiments,” which may each refer to one or more ofthe same or different embodiments. Furthermore, the terms “comprising,”“including,” “having,” and the like, as used with respect to embodimentsof the present disclosure, are synonymous.

Example embodiments may be described as a process depicted as aflowchart, a flow diagram, a data flow diagram, a structure diagram, ora block diagram. Although a flowchart may describe the operations as asequential process, many of the operations may be performed in parallel,concurrently, or simultaneously. In addition, the order of theoperations may be re-arranged. A process may be terminated when itsoperations are completed, but may also have additional steps notincluded in the figure(s). A process may correspond to a method, afunction, a procedure, a subroutine, a subprogram, and the like. When aprocess corresponds to a function, its termination may correspond to areturn of the function to the calling function and/or the main function.

Example embodiments may be described in the general context ofcomputer-executable instructions, such as program code, softwaremodules, and/or functional processes, being executed by one or more ofthe aforementioned circuitry. The program code, software modules, and/orfunctional processes may include routines, programs, objects,components, data structures, etc., that perform particular tasks orimplement particular data types. The program code, software modules,and/or functional processes discussed herein may be implemented usingexisting hardware in existing communication networks. For example,program code, software modules, and/or functional processes discussedherein may be implemented using existing hardware at existing networkelements or control nodes.

As used herein, the term “circuitry” refers to, is part of, or includeshardware components such as an electronic circuit, a logic circuit, aprocessor (shared, dedicated, or group) and/or memory (shared,dedicated, or group), an Application Specific Integrated Circuit (ASIC),a field-programmable device (FPD), (for example, a field-programmablegate array (FPGA), a programmable logic device (PLD), a complex PLD(CPLD), a high-capacity PLD (HCPLD), a structured ASIC, or aprogrammable System on Chip (SoC)), digital signal processors (DSPs),etc., that are configured to provide the described functionality. Insome embodiments, the circuitry may execute one or more software orfirmware programs to provide at least some of the describedfunctionality.

As used herein, the term “processor circuitry” may refer to, is part of,or includes circuitry capable of sequentially and automatically carryingout a sequence of arithmetic or logical operations; recording, storing,and/or transferring digital data. The term “processor circuitry” mayrefer to one or more application processors, one or more basebandprocessors, a physical central processing unit (CPU), a single-coreprocessor, a dual-core processor, a triple-core processor, a quad-coreprocessor, and/or any other device capable of executing or otherwiseoperating computer-executable instructions, such as program code,software modules, and/or functional processes.

As used herein, the term “interface circuitry” may refer to, is part of,or includes circuitry providing for the exchange of information betweentwo or more components or devices. The term “interface circuitry” mayrefer to one or more hardware interfaces (for example, buses,input/output (I/O) interfaces, peripheral component interfaces, networkinterface cards, and/or the like). As used herein, the terms“instantiate,” “instantiation,” and the like may refer to the creationof an instance, and an “instance” may refer to a concrete occurrence ofan object, which may occur, for example, during execution of programcode.

As used herein, the term “computer device” may describe any physicalhardware device capable of sequentially and automatically carrying out asequence of arithmetic or logical operations, equipped to record/storedata on a machine readable medium, and transmit and receive data fromone or more other devices in a communications network. A computer devicemay be considered synonymous to, and may hereafter be occasionallyreferred to, as a computer, computing platform, computing device, etc.The term “computer system” may include any type interconnectedelectronic devices, computer devices, or components thereof.Additionally, the term “computer system” and/or “system” may refer tovarious components of a computer that are communicatively coupled withone another. Furthermore, the term “computer system” and/or “system” mayrefer to multiple computer devices and/or multiple computing systemsthat are communicatively coupled with one another and configured toshare computing and/or networking resources. Examples of “computerdevices”, “computer systems”, etc. may include cellular phones or smartphones, feature phones, tablet personal computers, wearable computingdevices, an autonomous sensors, laptop computers, desktop personalcomputers, video game consoles, digital media players, handheldmessaging devices, personal data assistants, an electronic book readers,augmented reality devices, server computer devices (e.g., stand-alone,rack-mounted, blade, etc.), cloud computing services/systems, networkelements, in-vehicle infotainment (IVI), in-car entertainment (ICE)devices, an Instrument Cluster (IC), head-up display (HUD) devices,onboard diagnostic (OBD) devices, dashtop mobile equipment (DME), mobiledata terminals (MDTs), Electronic Engine Management System (EEMS),electronic/engine control units (ECUs), electronic/engine controlmodules (ECMs), embedded systems, microcontrollers, control modules,engine management systems (EMS), networked or “smart” appliances,machine-type communications (MTC) devices, machine-to-machine (M2M),Internet of Things (IoT) devices, and/or any other like electronicdevices. Moreover, the term “vehicle-embedded computer device” may referto any computer device and/or computer system physically mounted on,built in, or otherwise embedded in a vehicle.

As used herein, the term “network element” may be considered synonymousto and/or referred to as a networked computer, networking hardware,network equipment, router, switch, hub, bridge, radio networkcontroller, radio access network device, gateway, server, and/or anyother like device. The term “network element” may describe a physicalcomputing device of a wired or wireless communication network and beconfigured to host a virtual machine. Furthermore, the term “networkelement” may describe equipment that provides radio baseband functionsfor data and/or voice connectivity between a network and one or moreusers. The term “network element” may be considered synonymous to and/orreferred to as a “base station.” As used herein, the term “base station”may be considered synonymous to and/or referred to as a node B, anenhanced or evolved node B (eNB), next generation nodeB (gNB), basetransceiver station (BTS), access point (AP), roadside unit (RSU), etc.,and may describe equipment that provides the radio baseband functionsfor data and/or voice connectivity between a network and one or moreusers. As used herein, the terms “vehicle-to-vehicle” and “V2V” mayrefer to any communication involving a vehicle as a source ordestination of a message. Additionally, the terms “vehicle-to-vehicle”and “V2V” as used herein may also encompass or be equivalent tovehicle-to-infrastructure (V2I) communications, vehicle-to-network (V2N)communications, vehicle-to-pedestrian (V2P) communications, or V2Xcommunications.

As used herein, the term “channel” may refer to any transmission medium,either tangible or intangible, which is used to communicate data or adata stream. The term “channel” may be synonymous with and/or equivalentto “communications channel,” “data communications channel,”“transmission channel,” “data transmission channel,” “access channel,”“data access channel,” “link,” “data link,” “carrier,” “radiofrequencycarrier,” and/or any other like term denoting a pathway or mediumthrough which data is communicated. Additionally, the term “link” mayrefer to a connection between two devices through a Radio AccessTechnology (RAT) for the purpose of transmitting and receivinginformation.

Referring to the figures, FIG. 1 illustrates an arrangement 100 in whichvarious embodiments may be practiced. Arrangement 100 includes augmentedreality (AR) platform 105, network 150, local network 155, and one ormore servers 145. The local network 155 may include a user 108 with awearable device 110 and an environment 115. The environment 115 mayinclude sensor array 120, output array 122, and objects 131.

The AR platform 105 may be one or more computer devices capable ofdetecting motions and/or objects, generating virtual bindings and/orvirtual objects based on the detected objects and/or motions, and causethe virtual bindings and virtual objects to be displayed within theenvironment 115 according to the various embodiments herein. In thisregard, the AR platform 105 may include processor(s), memory device(s),communication device(s), and/or other like components to perform thefunctionality of the embodiments herein. An example implementation ofthe AR platform 105 is shown and described with regard to FIG. 2.

The AR platform 105 may detect (e.g., utilizing data obtained fromsensors of the sensor array 120 and/or wearable device 110) one or moreuser interactions to create virtual bindings 135A-B, connect orassociate the bindings 135A-B to physical (“real”) object 130P (e.g.,physical object 131P), virtual objects 131V, and/or instrumented object131I. The user interactions may include air and touch gestures performedby user 108 and/or voice commands issued by the user 108. For example,the user 108 may use voice commands to assign attributes to a binding135 or to choose events and physics for specific sessions and setups,and the user 108 may use gestures to select and manipulate a binding135.

In the example of FIG. 1, the user 108, via interaction with AR platform105, created virtual binding 135A and connected or associated thebinding 135A with virtual object 131V-1 and physical object 131P.Additionally, the user 108, via interaction with AR platform 105,created virtual binding 135B and connected or associated the binding135B with virtual object 131V-2 and instrumented object 131I. Thevirtual bindings 135 may take on attributes and/or perform actions basedon detected physical/instrumented objects. In embodiments, theattributes and/or actions may be based on a context of the environment115, other objects and bindings in the environment, and/or a relationbetween various objects and bindings in the environment 115. The“context” of the environment 115 may refer to interrelated conditions orcircumstances in which a setting or arrangement of objects 131 in theenvironment 115 exists or occurs, and may be based on various events,conditions, criteria, etc. that preceded that setting or arrangement.

The AR platform 105 may control sensors of sensor array 120 to recognizethe objects 131 and cause the bindings 135 to behave with appropriatephysical attributes (e.g., according to the laws of physics) and othermeaningful responses. The AR platform 105 may also interpret usergestures (including air and touch gestures), voice commands, and/orother user inputs to assign attributes to the bindings. For example,when the user 108 performs a pulling or dragging gesture to pull on abinding 135, the AR platform 105 may assign various physical propertiesto the binding 135, such as elasticity, stiffness, hardness, fluidity,color, etc. based on the environmental context. In another, example, theuser 108 may perform a drawing type gesture, for example, using afinger, to draw a specific type of binding 135, such as a binding with aspecific shape or size. Additionally, the AR platform 105 maycommunicate with instrumented objects 131I based on user gestures, etc.such that the instrumented objects 131I perform actions corresponding tothe user gestures.

The environment 115 may be any type of setting or area in/on which theuser 108 may interact with physical, instrumented, and virtual objects.In some embodiments, the environment 115 may be a table, a physicalplatform, or other like furniture object, and various virtual images maybe projected onto a surface of the table, the physical platform, etc. Insome embodiments, the environment 115 may be a room, or an area within aroom, into which virtual images may be projected (e.g., for providing atouchless user interface (TUI)). In some embodiments, the environment115 may be a table-top display device where various virtual images aredisplayed by projecting images/light upwards toward the user. In oneexample, the environment 115 may include a holographic display systemthat that utilizes light diffraction to create a virtualthree-dimensional (3D) image of one or more virtual objects 131V. Inalternative embodiments, the user 108 may use AR headgear, monocular orbinocular optical head-mounted display (OHMD), a 3D television with 3Dglasses, or the like, to view the environment 115 rather than using atable or object-based type of environment as discussed previously. TheAR platform 105 and the environment 115 may be collectively referred toas an “AR system” or the like.

As shown, the environment 115 may include the sensor array 120 andoutput array 122. The sensor array 120 may include one or more sensors(e.g., sensors 220 shown and described with regard to FIG. 2), which maybe devices capable of sensing, detecting, capturing, measuring, eventsor changes in the physical environment 115, and convert the sensedevents or environmental changes into a signal and/or sensor data, whichcan be read by a computing device. The sensors of the sensor array 120may include, inter alia, one or more image/motion detection/capturedevice(s) (e.g., cameras, depth sensors, infrared cameras, etc.),accelerometer(s), gyroscope(s), gravimeter(s), magnetometer(s),proximity sensor(s), ambient light sensor(s), microphone(s), pressuresensor(s), a capacitive surface, a conduction detector, a detector ofelectromagnetic energy (such as radiofrequency (RF), infrared, visiblelight, etc.), a vibration detector, a proximity detector, a fiducialmarker detector, and the like. In some implementations, the sensor array120 may include a stereo camera system comprising two or more cameraswhose relation to one another is known, and/or a depth sensor/camera or3D scanner system (e.g., a resonant MEMS micro mirror device, amicro-opto-electromechanical system (MOEMS), a time of flight (ToF)camera/sensor, etc.) that uses structured light, ToF, and/or infraredlaser projection, triangulation techniques to scan and analyze areal-world object or environment to collect data on its shape andpossibly its appearance (e.g., color, texture, etc.). In one exampleimplementation, the sensor array 120 may comprise a single housing witha stereo camera system with onboard image processing circuitry, wherethe image processing circuitry may measure the distance between pixelson any objects 131 that fall within the fields of view of the cameras ofthe stereo camera system. Examples of the sensor array may include anIntel® RealSense™ Depth Camera, a Microsoft® Kinect™ sensor, a StructureSensor by Occipital, Inc., a Google® Tango device, a Motion Contrast 3DScanning (MC3D) camera, or some other like devices.

Sensors of sensor array 120 may detect, for example, digital or physicalcharacteristics of physical objects 131P and/or instrumented objects131I in the environment 115. These characteristics may comprise, forexample, a time, an orientation, a position, a rate of change inposition/orientation, a relative position, encoded information, color,shape, size, sound, materials, or other like physical properties. Theposition, orientation, and rate of change in position/orientation may berelative to the sensors, relative to other objects (e.g. physical,virtual, or instrumented objects), or relative to an external frame ofreference. In various embodiments, the detected characteristics may beused by the AR platform 105 to emulate various physicalcharacteristics/properties of the bindings 135 and/or virtual objects131V. The detected characteristics may also be used to trigger events ortransition a state of play, such as by generating and displayingadditional or alternative virtual objects 131V or instructinginstrumented objects 131I to perform one or more actions. Inembodiments, the characteristics may be stored by the AR platform 105 asobject profile 322 and/or tracking history 323 records (see e.g., FIG.3).

Output array 120 may include one or more devices that are configured toconvert electronically generated signals/data into a medium that iscomprehendible by human sensory systems. Various output modalities maybe used, including visual, haptics, audio, and chemical. For example,user interactions (e.g., gestures or voice commands) on bindings 135 maycause the AR platform 105 to generated and display various images, emitsound effects or haptic vibrations on objects 131 and surfaces of theenvironment 115, and/or emit a gas, chemical scents, etc. Inembodiments, the output devices of the output array 120 may includedisplay devices, audio output devices, tactile output devices, chemicalemitting device, and the like. The display devices may be any type ofoutput device that is able to present information in a visual form basedon received electrical signals. In most embodiments, the display devicemay be an image or video projector that may project images or movingimages onto a display surface of environment 115 based on receivedsignals (e.g., picture generating units (PGUs) 230 and holographicoptical elements (HOEs) 231 shown and described with regard to FIG. 2).In other embodiments, the display devices may include monitors, such asLight Emitting Diode (LED) display devices, organic LED (OLED) displaydevices, Liquid Crystal Display (LCD) devices, quantum dot displaydevices, and/or the like. The display devices may be used to display oneor more images associated with virtual objects 131V-1 and 131V-2. Thetactile output devices may include electro-mechanical component array(EMCs) (e.g., EMCs 222 shown and described with regard to FIG. 2), maycontrol the environment 115 and/or the objects 131 in the environment115.

Network 150 and local network 155 may comprise computers, networkconnections among the computers, and software routines to enablecommunication between the computers over network connections. In thisregard, the network 150 and local network 155 may each comprise one ormore network elements that may include one or more processors,communications systems (e.g., including network interface controllers,one or more transmitters/receivers connected to one or more antennas,etc.), and computer readable media. Examples of such network elementsmay include wireless access points (WAPs), a home/business server (withor without radio frequency (RF) communications circuitry), a router, aswitch, a hub, a radio beacon, base stations, picocell or small cellbase stations, and/or any other like network device. Connection to thenetwork 150 or local network 155 may be via a wired or a wirelessconnection using the various communication protocols discussed infra. Asused herein, a wired or wireless communication protocol may refer to aset of standardized rules or instructions implemented by a communicationdevice/system to communicate with other devices, including instructionsfor packetizing/depacketizing data, modulating/demodulating signals,implementation of protocols stacks, and the like. More than one networkmay be involved in a communication session between the illustrateddevices. Connection to the network 150 and/or local network 155 mayrequire that the computers execute software routines which enable, forexample, the seven layers of the OSI model of computer networking orequivalent in a wireless (cellular) phone network

Network 150 may be used to enable relatively long-range communicationsuch as, for example, between the one or more servers 145 and ARplatform 105 or a component within local network 155. The network 150may represent the Internet, one or more cellular networks, a local areanetwork (LAN) or a wide area network (WAN) including proprietary and/orenterprise networks, Transfer Control Protocol (TCP)/Internet Protocol(IP)-based network, or combinations thereof. Examples of such networksand/or protocols are discussed infra with regard to FIG. 2. In suchembodiments, the network 150 may be associated with network operator whoowns or controls equipment and other elements necessary to providenetwork-related services, such as one or more base stations or accesspoints, one or more servers for routing digital data or telephone calls(for example, a core network or backbone network), etc.

Local network 155 may be used to enable relatively short-rangecommunication such as, for example, between AR platform 105 and sensorsof sensor array 120, output devices of output array 122, instrumentedobjects 131I, wearable device 110, or the like. The local network 155may represent a short-range network, such as person-to-person (P2P) orpersonal area network (PAN) (e.g., IEEE 802.15.4 based protocolsincluding ZigBee, IPv6 over Low power Wireless Personal Area Networks(6LoWPAN), WirelessHART, MiWi, Thread, etc.; WiFi-direct; Bluetooth/BLEprotocols; ANT protocols; Z-Wave; LTE D2D or ProSe; UPnP; and the like),or could represent any other network or protocols as discussed herein.

The one or more servers 145 may be one or more hardware computingdevices that may include one or more systems and/or applications forproviding one or more services to users (e.g., user 108 and/or ARplatform 105) over a network (e.g., network 150). The one or moreservers 145 may include one or more processors, one or more memorydevices, one or more network interfaces, etc. Additionally, one or moreservers 145 may be a single physical hardware device, or may bephysically or logically connected with other network devices. Moreover,one or more servers 145 may be connected to, or otherwise associatedwith one or more data storage devices (not shown). The server(s) 145 mayinclude an operating system (OS) that may provide executable programinstructions for the general administration and operation of servers,and may include a computer-readable medium storing instructions that,when executed by a processor of the servers, may allow the servers toperform their intended functions. Suitable implementations for the OSand general functionality of servers are known or commerciallyavailable, and are readily implemented by persons having ordinary skillin the art.

In embodiments, the services provided by the server(s) 145 may includeaccounting for, or otherwise operating an immersive gaming experience.For example, the services may include providing data to be used forgenerating virtual objects 131V, bindings 135, and/or instructions forinstrumented objects 131I, as well as and manipulating theobjects/bindings based on user interactions, object/binding attributes,context, gameplay rules, subscription data, and/or other like criteriaor parameters. The services may include creating, storing, andaltering/adjusting user profiles associated with a user 108, games, etc.The user profiles may indicate game criteria or parameters associatedwith the user in connection with a game, or in relation to multipleusers playing one or more games. The services may also include trackingor account for gameplay points/property and/or gameplay effects, such asvirtual currency/property/points tallies (including points, health,damage loss, power levels, “magical power”, etc.), a virtual or physicalon/off, open/close, and/or lock/unlock indication, physical dispenseamounts, virtual dispense amounts, and/or the like. The game playproperties may be represented as numerical values, character strings,etc. that is/are accounted for by the server(s) 145, and may increase ordecrease based on various factors such has the passage of time, time ofday and/or date, completing tasks, positions/orientations (or changes inpositions/orientations) of various objects in the environment 115,relation (or changes in relations) of various objects with one anotherin the environment 115, and/or the like. The server(s) 145 may alsoperform or facilitate user setup and play registration, includingassociating one or more physical, virtual, and/or instrumented objectswith authorized users 108, initiate and control software and/or firmwareupdates of the elements or devices within the environment 115, recordthe results associated with one or more games, provide requested userauthentication credentials, provide content management, provide userinterfaces and/or control elements for setting up new games and/ormodifying existing games, and/or perform computationally intensive tasksfor the AR platform 105 and/or the components/devices within environment115. In some embodiments, some or all of the functionality performed bythe server(s) 145 may be implemented by the AR platform 105, or viceversa.

The virtual objects 131V and bindings 135 may be one or more images,animations, video clips, holograms, holographic optical elements, etc.that are displayed based on user interactions (e.g., gestures, voicecommands, etc.), various physical objects 131P in the environment 115,various instrumented objects 131I in the environment 115, various andbindings 135 in the environment 115, one or more in-game criteria,and/or the like. The virtual objects 131I and bindings 135 may begenerated and rendered (i.e., displayed and/or projected) by an imagegeneration module and/or a rendering module (e.g., AR engine 331 and/orAR renderer 333 as described with regard to FIG. 3) according to one ormore known methods for generating computer graphics. For instance, theimages associated with virtual objects 131I and bindings 135 may be atwo dimensional (2D) pixel graphic generated using a raster imagingapplication, a sprite graphic generated using a cascading style sheet(CSS), a vector graphic generated according to the scalable vectorgraphics (SVG) standard, a three dimensional (3D) graphic generatedusing one or more known 3D modeling and/or 3D rendering applications,computer-generated imagery (CGI), etc. In some implementations, thevirtual objects 131I may be described using Extensible Markup Language(XML), JavaScript Object Notation (JSON), Augmented Reality MarkupLanguage (ARML), and/or other like data structure or language, orvariants thereof. As alluded to previously, the virtual objects 131I andbindings 135 may be generated and rendered based on a 2D or 3D modelaccording to one or more known methods, such as by obtaining object dataand/or image data from one or more data packets, processing the objectdata and/or image data into a scene file taking into account geometryinformation, viewpoint information, scene, information, textureinformation, lighting information, shading information, featureinformation, visual assets information, anchors (e.g., geometries,trackables, relative-to, screen anchors, etc.), and/or the like, andoutputting the processed data to a digital image file, a raster imagefile, and the like. The hardware devices used by the image generationmodule and the rendering module e.g., AR engine 331 and/or AR renderer333 as described with regard to FIG. 3) may include a graphicsprocessing unit (GPU) and/or any other like hardware device that isconfigured to perform complex rendering calculations. The generatedimages may be displayed according to one or more known video mapping orprojection mapping methods, one or more known projection methods, and/orany other like known method for displaying images.

Although both virtual objects 131I and bindings 135 may be generatedand/or rendered in a same or similar manner, the virtual objects 131Iand bindings 135 may be different in some respects. In embodiments, thevirtual objects 131I may be digital images (including real and virtualimages) that represent real-world objects, whereas bindings 135 may bedigital images (including real and virtual images) that bind to or areotherwise associated with one or more objects (including physicalobjects 131P, virtual objects 131V, instrumented objects 131I, and/orother bindings 135) to one or more other objects or to nothing at all.In embodiments, the bindings may take on attributes based on the otherobjects in the environment 115 (or relations therebetween), userinteractions, or the like as detected by the sensors of the senor array120. In embodiments, the attributes may define (or may be used todefine) various physical attributes that a binding 135 is to emulate.

In embodiments, the AR platform 105 may enable the user 106 to selectone of a plurality of virtual binding types with different physicsaspects, allowing the user 108 to select behaviors and appearances ofthe binding 135. In one example, the AR platform 105 may project theplurality of binding types, and the user 108 may perform one or moregestures to select a desired one of the binding types. Moreover,multiple bindings 135 and objects 131 may be used simultaneously bymultiple users 108.

The physical objects 131P may be any type of object that is physicallypresent in the environment 115, and may include, for example, cards(e.g., playing cards, collectable cards, trading cards, etc.), analogtoys (e.g., action figures/dolls, construction sets or construction setpieces, toy vehicles, puzzles or puzzle pieces, board games, etc.),furniture, office supplies (e.g., writing implements, hole punches,stamps, staplers, staples, paper clips, etc.), sports equipment (e.g.,baseball bats, baseball gloves, golf clubs, golf balls, tennis rackets,tennis balls, etc.), and/or other like physical objects. Although theaforementioned examples are inanimate objects, embodiments herein mayalso be applicable to animate objects (e.g., robots or other mechanicaldevices) and/or living organisms (e.g., dogs, cats, or other types ofpets).

Instrumented objects 131I may be physical objects that are embedded withhardware components and/or electromechanical devices (EMCs) that enablethe instrumented objects 131I to perform one or more actions, andoptionally to capture/record and communicate data associated with anevent one or more other devices over a network with little or no userintervention. In embodiments, an instrumented object 131I may comprisean computer device/system embedded in some other device or object, suchas robots, smart toys, smart appliances, and/or the like, and mayinclude microcontrollers or microprocessors, memory/storage devices,communication circuitry (e.g., transceiver(s), modem, antenna elements,etc.), sensors/meters, image capture devices, microphones, lightemitting devices, audio emitting devices, image/video playback devices,EMCs, and/or other like components that may be the same or similar tothose discussed herein.

The instrumented objects 131I may store program code (or have FPDspreconfigured with appropriate logic blocks/fabric or the like) toperform one or more actions mentioned previously. According to variousembodiments, the communications circuitry of the instrumented objects131I may receive one or more messages from the AR platform 105, whichmay include instructions to perform indicated actions of the one or moreactions. The instructions may be based on the user interactions detectedby the AR platform 105. This may allow the instrumented objects 131I tointeract with the user 108 and/or other objects (e.g., physical,virtual, or instrumented objects) based on the various user interactionsin the environment 115. In embodiments where the instrumented objects131I may capture, store/record, and communicate data associated with anevent, the event data captured by instrumented objects 131I may be usedto augment or supplement the environment modeling mechanisms that isdone using the sensor data from the sensors of the sensor array 120.

The user 108 may be an operator of a AR platform 105 and may interactwith the various objects 131 in the environment 115. In someimplementations, the user 108 may affix or couple the wearable 110 tohis/her body. As shown, the wearable 110 is coupled with a wrist of theuser 108; however, the user 108 may affix multiple wearables 110 tohis/her body, such as a wearable 110 for each wrist of the user 108, awearable 110 coupled with a torso of the user 108, or the like. Sensordata captured by the wearable 110 may be used to determine positionand/or orientation information and/or body motions of one or more bodyparts of the user 108. Additionally, although FIG. 1 only shows a singleuser 108, the embodiments herein may allow multiple user 108 to interactwith multiple objects 131 and multiple bindings 135.

Wearable device 110 (or “wearable 110”) may be an inertial sensor unit(IMU) or some other suitable device that is embedded with hardware thatenable measurement and/or detection of motion, acceleration, rotation,and/or an orientation of the wearable 110. The wearable 110 may includeone or more microelectromechanical system (MEMS) or sensors, such asaccelerometers, gyroscopes, magnetometers, and/or other like sensors.The one or more MEMS/sensors may be configured to determine a magnitudeand direction of a velocity, acceleration, and/or motion of the wearable110, and convert the velocity, acceleration, and/or motion of thewearable 110 into position and/or orientation information. The changesin the positions and/or orientations of the wearable 110 may beindicative of a biomechanical motion performed by a user 108 of thewearable 110. The one or more MEMS/sensors may be configured to detectthe biomechanical motion as sensor data. The sensor data may include orotherwise indicate one or more spatial coordinates (or changes inspatial coordinates) for the positions and/or orientations of thewearable 110. The sensor data may then be passed to one or moreprocessors and/or a sensor hub of the wearable 110 to be converted intoa biomechanical sequence, sensor coordinates, and/or any other type offormat of analyzed, processed, or formatted data. Furthermore, thewearable 110 may track a timing of the biomechanical motion bytimestamping the sensor data as it is collected and/or processed.Wearable 110 may also include, or be coupled with communicationscircuitry that enables the wearable 110 to communicate with one or moreother devices (e.g., AR platform 105 and/or server(s) 145) over anetwork (e.g., network 150 or local network 155) via a wired or wirelessnetwork with little or no user intervention. In this regard, one or morememory devices, and one or more processors. As examples, the wearable110 may be any type of wearable computer device or wearable technology,such as a smart watch, fitness or activity tracker, a sensor systemembedded in equipment or clothing, a telemetry system, and/or the like.In embodiments where the user 108 uses a wearable 110, the sensor datacaptured by the wearable 110 may be used to augment or supplement thegesture detection mechanisms that use the sensor data from the sensorsof the sensor array 120.

Although FIG. 1 shows various devices and/or components, according tovarious embodiments, any number of computing devices (clients andservers), sensors, output devices, users, and/or of databases (notshown) may be present. Additionally, some or all of the devicescomponents shown by FIG. 1 may be separate and positioned in variousarrangements, collocated with one another, reside on one physicalhardware device or otherwise fully integrated with one another,implemented as virtual machines, and/or set up in a multitude ofimplementations, arrangements, and/or environments. Thus, the depictionof the illustrative embodiments of FIG. 1 should be taken as beingillustrative in nature, and not limited to the scope of the disclosure.

FIG. 2 illustrates an example implementation of a computer device 200,in accordance with various embodiments. FIG. 2 shows a block diagram ofan example of components that may be present in computer device 200. Thecomputer device 200 may include any combinations of the components shownFIG. 2. The components may be implemented as integrated circuits (ICs)or portions thereof, discrete electronic devices, or other modules,logic, hardware, software, firmware, middleware or a combination thereofadapted in the computer device 200, or as components otherwiseincorporated within a chassis of a larger system.

The computer device 200 may be an embedded system or any other type ofcomputer device discussed herein. In the example shown by FIG. 2, thecomputer device 200 may be employed in or as an augmented reality (AR)platform 105 or other like device/system as discussed herein. AlthoughFIG. 2 shows the computer device 200 as implemented in AR platform 105,computer device 200 or a computer device that is similar to computerdevice 200 may be implemented in other devices discussed herein, such asthe instrumented object 131I and the one or more servers 145. In anotherexample, the computer device 200 may be a separate and dedicated and/orspecial-purpose computer device designed specifically to carry out theAR solutions of the embodiments discussed herein.

Processor(s) 202 (also referred to as “processor circuitry 202”) may beone or more processing elements configured to perform basicarithmetical, logical, and input/output operations by carrying outinstructions. Processor circuitry 202 may be implemented as a standalonesystem/device/package or as part of an existing system/device/package ofthe AR system 110. The processor circuitry 202 may be one or moremicroprocessors, one or more single-core processors, one or moremulti-core processors, one or more multithreaded processors, one or moreGPUs, one or more ultra-low voltage processors, one or more embeddedprocessors, one or more DSPs, one or more FPDs (hardware accelerators)such as FPGAs, structured ASICs, programmable SoCs (PSoCs), etc., and/orother processor or processing/controlling circuit. The processorcircuitry 202 may be a part of a system on a chip (SoC) in which theprocessor circuitry 202 and other components discussed herein are formedinto a single IC or a single package. As examples, the processorcircuitry 202 may include one or more Intel Pentium®, Core®, Xeon®,Atom®, or Core M® processor(s); Advanced Micro Devices (AMD) AcceleratedProcessing Units (APUs), Epyc®, or Ryzen® processors; Apple Inc. Aseries, S series, W series, etc. processor(s); Qualcomm snapdragon®processor(s); Samsung Exynos® processor(s); and/or the like.

In embodiments, the processor circuitry 202 may include a sensor hub,which may act as a coprocessor by processing data obtained from thesensors 220. The sensor hub may include circuitry configured tointegrate data obtained from each of the sensors 220 by performingarithmetical, logical, and input/output operations. In embodiments, thesensor hub may capable of timestamping obtained sensor data, providingsensor data to the processor circuitry 202 in response to a query forsuch data, buffering sensor data, continuously streaming sensor data tothe processor circuitry 202 including independent streams for eachsensor 322, reporting sensor data based upon predefined thresholds orconditions/triggers, and/or other like data processing functions.

Memory 204 (also referred to as “memory circuitry 204” or the like) maybe circuitry configured to store data or logic for operating thecomputer device 200. Memory circuitry 204 may include number of memorydevices may be used to provide for a given amount of system memory. Asexamples, the memory circuitry 204 can be any suitable type, numberand/or combination of volatile memory devices (e.g., random accessmemory (RAM), dynamic RAM (DRAM), static RAM (SAM), etc.) and/ornon-volatile memory devices (e.g., read-only memory (ROM), erasableprogrammable read-only memory (EPROM), electrically erasableprogrammable read-only memory (EEPROM), flash memory, antifuses, etc.)that may be configured in any suitable implementation as are known. Invarious implementations, individual memory devices may be formed of anynumber of different package types, such as single die package (SDP),dual die package (DDP) or quad die package (Q17P), dual inline memorymodules (DIMMs) such as microDIMMs or MiniDIMMs, and/or any other likememory devices. To provide for persistent storage of information such asdata, applications, operating systems and so forth, the memory circuitry204 may include one or more mass-storage devices, such as a solid statedisk drive (SSDD); flash memory cards, such as SD cards, microSD cards,xD picture cards, and the like, and USB flash drives; on-die memory orregisters associated with the processor circuitry 202 (for example, inlow power implementations); a micro hard disk drive (HDD); threedimensional cross-point (3D XPOINT) memories from Intel® and Micron®,etc.

Where FPDs are used, the processor circuitry 202 and memory circuitry204 (and/or device storage circuitry 208) may comprise logic blocks orlogic fabric, memory cells, input/output (I/O) blocks, and otherinterconnected resources that may be programmed to perform variousfunctions of the example embodiments discussed herein. The memory cellsmay be used to store data in lookup-tables (LUTs) that are used by theprocessor circuitry 202 to implement various logic functions. The memorycells may include any combination of various levels of memory/storageincluding, but not limited to, EPROM, EEPROM, flash memory, SRAM,anti-fuses, etc.

Data storage circuitry 208 (also referred to as “storage circuitry 208”or the like), with shared or respective controllers, may provide forpersistent storage of information such as modules 209, operatingsystems, etc. The storage circuitry 208 may be implemented as solidstate drives (SSDs); solid state disk drive (SSDD); serial AT attachment(SATA) storage devices (e.g., SATA SSDs); flash drives; flash memorycards, such as SD cards, microSD cards, xD picture cards, and the like,and USB flash drives; three-dimensional cross-point (3D Xpoint) memorydevices; on-die memory or registers associated with the processorcircuitry 202; hard disk drives (HDDs); micro HDDs; resistance changememories; phase change memories; holographic memories; or chemicalmemories; among others. As shown, the storage circuitry 208 is includedin the computer device 200; however, in other embodiments, storagecircuitry 208 may be implemented as one or more separate devices thatare mounted in AR platform 105 separate from the other elements ofcomputer device 200.

In some embodiments, the storage circuitry 208 may include an operatingsystem (OS) (not shown), which may be a general purpose operating systemor an operating system specifically written for and tailored to thecomputer device 200. The OS may include one or more drivers, libraries,and/or application programming interfaces (APIs), which provide programcode and/or software components for modules 209 and/or control systemconfigurations to control and/or obtain/process data from one or moresensors 220 and/or EMCs 222.

The modules 209 may be software modules/components used to performvarious functions of the computer device 200 and/or to carry outfunctions of the example embodiments discussed herein. In embodimentswhere the processor circuitry 202 and memory circuitry 204 includeshardware accelerators (e.g., FPGA cells) as well as processor cores, thehardware accelerators (e.g., the FPGA cells) may be pre-configured(e.g., with appropriate bit streams, logic blocks/fabric, etc.) with thelogic to perform some functions of the embodiments herein (in lieu ofemployment of programming instructions to be executed by the processorcore(s)). For example, the modules 209 may comprise logic for thecorresponding entities discussed with regard to FIG. 3, namely, modelingengine 301, user input analysis engine 311, object recognition module321, AR engine 331, context engine 341, and content/applications 351.These entities are discussed in more detail with regard to FIG. 3.

The components of computer device 200 and/or AR system 110 maycommunicate with one another over the bus 206. The bus 206 may includeany number of technologies, such as a Local Interconnect Network (LIN);industry standard architecture (ISA); extended ISA (EISA); PCI; PCIextended (PCIx); PCIe; an Inter-Integrated Circuit (I2C) bus; a ParallelSmall Computer System Interface (SPI) bus; Common ApplicationProgramming Interface (CAPI); point to point interfaces; a power bus; aproprietary bus, for example, Intel® Ultra Path Interface (UPI), Intel®Accelerator Link (IAL), or some other proprietary bus used in a SoCbased interface; or any number of other technologies. In someembodiments, bus 206 may be a controller area network (CAN) bus system,a Time-Trigger Protocol (TTP) system, or a FlexRay system, which mayallow various devices (e.g., sensors 220, EMCs 222, etc.) to communicatewith one another using messages or frames.

The communications circuitry 305 may include circuitry for communicatingwith a wireless network or wired network. For example, the communicationcircuitry 305 may include transceiver (Tx) 211 and network interfacecontroller (NIC) 212. Communications circuitry 305 may include one ormore processors (e.g., baseband processors, modems, etc.) that arededicated to a particular wireless communication protocol.

NIC 212 may be included to provide a wired communication link to thenetwork 150 and/or other devices. The wired communication may provide anEthernet connection, an Ethernet-over-USB, and/or the like, or may bebased on other types of networks, such as DeviceNet, ControlNet, DataHighway+, PROFIBUS, or PROFINET, among many others. An additional NIC212 may be included to allow connect to a second network (not shown) orother devices, for example, a first NIC 212 providing communications tothe network 150 over Ethernet, and a second NIC 212 providingcommunications to other devices over another type of network, such as apersonal area network (PAN) including a personal computer (PC) device.In some embodiments, the various components of the environment 115, suchas sensors 220, EMCs 222, PGUs 230, etc. may be connected to the system110 via the NIC 212 as discussed above rather than via the I/O circuitry218 as discussed infra.

The Tx 211 may include one or more radios to wirelessly communicate withthe network 150 and/or other devices. The Tx 211 may include hardwaredevices that enable communication with wired networks and/or otherdevices using modulated electromagnetic radiation through a solid ornon-solid medium. Such hardware devices may include switches, filters,amplifiers, antenna elements, and the like to facilitate thecommunications over the air (OTA) by generating or otherwise producingradio waves to transmit data to one or more other devices, andconverting received signals into usable information, such as digitaldata, which may be provided to one or more other components of computerdevice 200. In some embodiments, the various components of theenvironment 115, such as sensors 220, EMCs 222, PGUs 230, etc. may beconnected to the system 110 via the Tx 211 as discussed above ratherthan via the I/O circuitry 218 as discussed infra. In one example, oneor more sensors 220 may be coupled with system 110 via a short rangecommunication protocol, such as BLE or the like. In another example, thePGUs may be coupled with the system 110 via a wireless connection (e.g.,via Tx 211 or the like) and operate in conjunction with one or moreremote display protocols, such as the wireless gigabit alliance (WiGiG)protocol, the remote desktop protocol (RDP), PC-over-IP (PCoIP)protocol, the high-definition experience (HDX) protocol, and/or otherlike remote display protocols.

The Tx 211 may include one or multiple radios that are compatible withany number of 3GPP (Third Generation Partnership Project)specifications, notably Long Term Evolution (LTE), Long TermEvolution-Advanced (LTE-A), Long Term Evolution-Advanced Pro (LTE-APro), and Fifth Generation (5G) New Radio (NR). It can be noted thatradios compatible with any number of other fixed, mobile, or satellitecommunication technologies and standards may be selected. These mayinclude, for example, any Cellular Wide Area radio communicationtechnology, which may include e.g. a 5G communication systems, a GlobalSystem for Mobile Communications (GSM) radio communication technology, aGeneral Packet Radio Service (GPRS) radio communication technology, oran Enhanced Data Rates for GSM Evolution (EDGE) radio communicationtechnology. Other Third Generation Partnership Project (3GPP) radiocommunication technology that may be used includes UMTS (UniversalMobile Telecommunications System), FOMA (Freedom of Multimedia Access),3GPP LTE (Long Term Evolution), 3GPP LTE Advanced (Long Term EvolutionAdvanced), 3GPP LTE Advanced Pro (Long Term Evolution Advanced Pro)),CDMA2000 (Code division multiple access 2000), CDPD (Cellular DigitalPacket Data), Mobitex, 3G (Third Generation), CSD (Circuit SwitchedData), HSCSD (High-Speed Circuit-Switched Data), UMTS (3G) (UniversalMobile Telecommunications System (Third Generation)), W-CDMA (UMTS)(Wideband Code Division Multiple Access (Universal MobileTelecommunications System)), HSPA (High Speed Packet Access), HSDPA(High-Speed Downlink Packet Access), HSUPA (High-Speed Uplink PacketAccess), HSPA+ (High Speed Packet Access Plus), UMTS-TDD (UniversalMobile Telecommunications System—Time-Division Duplex), TD-CDMA (TimeDivision—Code Division Multiple Access), TD-SCDMA (TimeDivision—Synchronous Code Division Multiple Access), 3GPP Rel. 8(Pre-4G) (3rd Generation Partnership Project Release 8 (Pre-4thGeneration)), 3GPP Rel. 9 (3rd Generation Partnership Project Release9), 3GPP Rel. 10 (3rd Generation Partnership Project Release 10), 3GPPRel. 11 (3rd Generation Partnership Project Release 11), 3GPP Rel. 12(3rd Generation Partnership Project Release 12), 3GPP Rel. 13 (3rdGeneration Partnership Project Release 13), 3GPP Rel. 14 (3rd GenerationPartnership Project Release 14), 3GPP LTE Extra, LTE Licensed-AssistedAccess (LAA), UTRA (UMTS Terrestrial Radio Access), E-UTRA (Evolved UMTSTerrestrial Radio Access), LTE Advanced (4G) (Long Term EvolutionAdvanced (4th Generation)), cdmaOne (2G), CDMA2000 (3G) (Code divisionmultiple access 2000 (Third generation)), EV-DO (Evolution-DataOptimized or Evolution-Data Only), AMPS (1G) (Advanced Mobile PhoneSystem (1st Generation)), TACS/ETACS (Total Access CommunicationSystem/Extended Total Access Communication System), D-AMPS (2G) (DigitalAMPS (2nd Generation)), PTT (Push-to-talk), MTS (Mobile TelephoneSystem), IMTS (Improved Mobile Telephone System), AMTS (Advanced MobileTelephone System), OLT (Norwegian for Offentlig Landmobil Telefoni,Public Land Mobile Telephony), MTD (Swedish abbreviation forMobiltelefonisystem D, or Mobile telephony system D), Autotel/PALM(Public Automated Land Mobile), ARP (Finnish for Autoradiopuhelin, “carradio phone”), NMT (Nordic Mobile Telephony), Hicap (High capacityversion of NTT (Nippon Telegraph and Telephone)), CDPD (Cellular DigitalPacket Data), Mobitex, DataTAC, iDEN (Integrated Digital EnhancedNetwork), PDC (Personal Digital Cellular), CSD (Circuit Switched Data),PHS (Personal Handy-phone System), WiDEN (Wideband Integrated DigitalEnhanced Network), iBurst, Unlicensed Mobile Access (UMA, also referredto as also referred to as 3GPP Generic Access Network, or GANstandard)), Wireless Gigabit Alliance (WiGig) standard, mmWave standardsin general (wireless systems operating at 10-90 GHz and above such asWiGig, IEEE 802.11ad, IEEE 802.11ay, and the like. In addition to thestandards listed above, any number of satellite uplink technologies maybe used for the uplink transceiver 711, including, for example, radioscompliant with standards issued by the ITU (InternationalTelecommunication Union), or the ETSI (European TelecommunicationsStandards Institute), among others. The examples provided herein arethus understood as being applicable to various other communicationtechnologies, both existing and not yet formulated. Implementations,components, and details of the aforementioned protocols may be thoseknown in the art and are omitted herein for the sake of brevity.

The input/output (I/O) interface 218 may include circuitry, such as anexternal expansion bus (e.g., Universal Serial Bus (USB), FireWire,Thunderbolt, PCl/PCIe/PCIx, etc.), used to connect computer device 200with external components/devices, such as sensors 220, EMCs 222, PGUs230, etc. I/O interface circuitry 218 may include any suitable interfacecontrollers and connectors to interconnect one or more of the processorcircuitry 202, memory circuitry 204, data storage circuitry 208,communication circuitry 305, and the other components of computer device200. The interface controllers may include, but are not limited to,memory controllers, storage controllers (e.g., redundant array ofindependent disk (RAID) controllers, baseboard management controllers(BMCs), input/output controllers, host controllers, etc. The connectorsmay include, for example, busses (e.g., bus 206), ports, slots, jumpers,interconnect modules, receptacles, modular connectors, etc. The I/Ocircuitry 218 may couple the system 110 with sensors 220, EMCs 320,PGUs, etc. via a wired connection, such as using USB, FireWire,Thunderbolt, RCA, a video graphics array (VGA), a digital visualinterface (DVI) and/or mini-DVI, a high-definition multimedia interface(HDMI), an S-Video, and/or the like. Although FIG. 2 shows that thesensors 220, EMCs 222, and PGUs 230 are coupled with the computer device200 via interface circuitry 218, in other embodiments, the sensors 220,EMCs 222, and PGUs 230 may be communicatively coupled with the computerdevice 200 via Tx 211, using short-range radio links, WiFi signaling, orthe like.

Sensors 220 may be any device configured to detect events orenvironmental changes, convert the detected events into electricalsignals and/or digital data, and transmit/send the signals/data to thecomputer device 200 and/or one or more EMCs 222. Some of the sensors 220may be sensors used for providing computer-generated sensory inputs inenvironment 115. Some of the sensors 220 may be sensors used for motionand/or object detection. Examples of such sensors 220 may include, interalia, charged-coupled devices (CCD), Complementarymetal-oxide-semiconductor (CMOS) active pixel sensors (APS), lens-lessimage capture devices/cameras, thermographic (infrared) cameras, LightImaging Detection And Ranging (LIDAR) systems, and/or the like. In someimplementations, the sensors 220 may include a lens-less image capturemechanism comprising an array of aperture elements, wherein lightpassing through the array of aperture elements define the pixels of animage. In embodiments, the motion detection sensors 220 may be coupledwith or associated with light generating devices, for example, one ormore infrared projectors to project a grid of infrared light onto ascene or environment 115, where an infrared camera may record reflectedinfrared light to compute depth information.

Some of the sensors 220 may be used for position and/or orientationdetection, ambient/environmental condition detection, and the like.Examples of such sensors 220 may include, inter alia,microelectromechanical systems (MEMS) with piezoelectric, piezoresistiveand/or capacitive components, which may be used to determineenvironmental conditions or location information related to the computerdevice 200. In embodiments, the MEMS may include 3-axis accelerometers,3-axis gyroscopes, and/or magnetometers. In some embodiments, thesensors 220 may also include one or more gravimeters, altimeters,barometers, proximity sensors (e.g., infrared radiation detector(s) andthe like), depth sensors, ambient light sensors, thermal sensors(thermometers), ultrasonic transceivers, and/or the like.

The EMCs 222 may be devices that allow computer device 200 to change astate, position, orientation, move, and/or control a mechanism orsystem. The EMCs 222 may include one or more switches; haptic outputdevices, such as actuators and/or motors (e.g., eccentric rotating mass(ERM) actuators, linear resonant actuator (LRA), piezoelectricactuators, servomechanisms, rotary motors, linear motors, and stepmotors, etc.), thrusters, projectile ejecting devices (e.g., usingspring loaded or compressed air/fluid), and/or the like. In embodiments,the EMCs 222 may comprise speakers, a digital rendering module(s) (e.g.,a physical object with a digital rendering module therein), and/oranother way to control an acoustic energy emission, an electromagneticradiation emission, an electric energy application, a magnetic field,and an acceleration or deceleration emitted or experienced by a physicalobject 130P, including by an instrumented object 131I. In embodiments,computer device 200 may be configured to operate one or more EMCs 222 bytransmitting/sending instructions or control signals to the EMCs 222based on detected user interactions or other like events.

Picture generation units (PGUs) 230 and optical elements (OEs) 232. ThePGUs 230 may generate light (e.g., based on digital images), which maybe directed and/or redirected to an OE 232 (e.g., a display surface).The digital images may be any type of content stored by the storagecircuitry 208, streamed from remote devices via the communicationcircuitry 205, and/or based on outputs from various sensors 220, EMCs222, and/or instrumented objects 131I. The generated light may becombined or overlapped with external (e.g., natural) light that is alsoredirected to the same OE 232. The OE 232 that combines the generatedlight with the external light may be referred to as a “combiner element”or “combiner.”

The PGUs 230 may be one or more electronic devices that create/generatedigital images to be directed to OEs 232. The PGUs 230 may be orcomprise a projector that may project still or moving images onto thesurface(s) of OEs 232 via one or more reflection surfaces (e.g.,mirrors) based on a received signal. The projector of each PGU 330 maybe an LED projector, a laser diode projector, a LCD projector, a digitallight processing (DLP) projector, a liquid crystal on silicon (LCoS)projector, and/or any other like projection device. The projector maycomprise a light source and various electronic devices (or electronicsystem) that may generate the images for display, such as one or moreprocessors/GPUs, one or more memory devices, and other like components.This may be done by converting the image into a signal for controllingthe light source to generate/output light of different colors andintensities. The projector may also comprise a combiner (also referredto as “combiner optic” and the like), which may combine different lightpaths into one light path to define a palette of colors. In someembodiments, the projector may comprise scanning mirrors that copy theimage pixel-by-pixel and then project the image for display. In someembodiments, the PGUs 230 may comprise a relay lens assembly and acombiner element (which may be different than the combiner of theprojector). The relay lens assembly may comprise one or more relaylenses, which re-image images from the projector into an intermediateimage that then reaches an OE 232 (e.g., the combiner element) through areflector.

The combiner element (as well as other OEs 232) may be a displaysurface, which may be fully or partially opaque or transparent, thatmixes the digital images output by the projector/PGUs 230 with viewedreal-world objects to facilitate augmented reality. One or more of theOEs 232 may be transmissive optical elements, where the transmitted beam(reference beam) hits the OE 232 and the diffracted beam(s) go throughthe OE 232. One or more OEs 232 may be reflective optical elements,where the transmitted beam (reference beam) hits the OE 232 and thediffracted beam(s) reflects off of the OE 232 (e.g., the reference beamand diffracted beams are on the same side of the OE 232). Inembodiments, the OEs 232 may be a holographic OE, and in someembodiments, the combiner element may be a hologram or holographic image(e.g., ether a transmissive HOE or reflective HOE).

Where HOEs 232 are used, one or more of the HOEs 232 may use waveguideholographic techniques to progressively extract a collimated imageguided by total internal reflection (TIR) in a waveguide pipe. Thewaveguide pipe may be a thin sheet of glass or plastic through which thegenerated light bounces to route the generated light to the viewer/user.In some embodiments, the HOEs 232 may utilize holographic diffractiongrating (e.g., Bragg diffraction grating) to provide the generated lightto the waveguide at a critical angle, which travels through thewaveguide. The light is steered toward the user/viewer by one or moreother HOEs 232 that utilize holographic diffraction grating. These HOEs232 may comprise grooved reflection gratings and/or a plurality oflayers of alternating refraction indexes (e.g., comprising liquidcrystals, photoresist substrate, etc.); the grooved reflection gratingsand/or the refractive index layers may provide constructive anddestructive interference and wavelet dispersion.

The battery 228 may power the computer device 200. In embodiments, thebattery 328 may be a lithium ion battery, a metal-air battery, such as azinc-air battery, an aluminum-air battery, a lithium-air battery, alithium polymer battery, and the like. The battery monitor 226 may beincluded in the computer device 200 to track/monitor various parametersof the battery 228, such as a state of charge (SoCh) of the battery 228,state of health (SoH), and the state of function (SoF) of the battery228. The battery monitor 226 may include a battery monitoring IC, whichmay communicate battery information to the processor circuitry 202 overthe bus 206.

Bus 206 may allow components of computer device 200 and/or AR system 110to communicate with one another. The bus 206 may include any number oftechnologies, such as a Local Interconnect Network (LIN); industrystandard architecture (ISA); extended ISA (EISA); Peripheral ComponentInterconnect Express (PCI); PCI extended (PCIx); PCI express (PCIe); anInter-Integrated Circuit (I2C) bus; a Parallel Small Computer SystemInterface (SPI) bus; point to point interfaces; a power bus; aproprietary bus, for example, used in a SoC based interface; or anynumber of other technologies. Suitable implementations and generalfunctionality of such bus systems are known, and are readily implementedby persons having ordinary skill in the art.

While not shown, various other devices may be present within, orconnected to, the computer device 200. For example, I/O devices, such asa display, a touchscreen, or keypad may be connected to the computerdevice 200 via bus 206 to accept input and display outputs. In anotherexample, the computer device 200 may include or be coupled withpositioning circuitry configured to determine coordinates based onsignals received from global navigation satellite system (GNSS)constellations. In another example, the communications circuitry 305 mayinclude a Universal Integrated Circuit Card (UICC), embedded UICC(eUICC), and/or other elements/components that may be used tocommunicate over one or more wireless networks.

FIG. 3 is a block diagram of a non-transitory, machine readable medium(NTMRM) 300 including code to direct a processor 302 to perform variousfunctions delineated by the embodiments discussed herein. Inembodiments, the non-transitory, machine readable medium 300 may beimplemented in an AR platform 105 and/or server(s) 145. The processor302 may access the non-transitory, machine readable medium 300 over abus 306. The processor 302 and bus 306 may be the same or similar asdescribed with respect to the processor 202 and bus 206 of FIG. 2,respectively. The non-transitory, machine readable medium 300 mayinclude devices described for the mass storage 208 of FIG. 2 or mayinclude optical disks, thumb drives, or any number of other hardwaredevices. In alternate embodiments, machine readable medium 300 may betransitory, e.g., signals. The NTMRM 300 may include code of a modelingengine 301 to direct the processor 302 to obtain first sensor data fromsensors 220 of the sensor array 120. The first sensor data may berepresentative of a physical environment (e.g., environment 115) and/orphysical objects 131P in the environment 115. In some embodiments, themodeling engine 301 may direct the processor 302 to track physicalobjects 131P in the environment 115 based on information of the firstsensor data. In some embodiments, the modeling engine 301 may direct theprocessor 302 to generate a three-dimensional (3D) model 303 of thephysical environment 115 for an AR environment based on the first sensordata. The 3D model 303 may be a suitable collection of data points in 3Dspace connected by various geometric shapes or entities and may includetextual information for various surfaces of the environment 115. The 3Dmodel 303 may be generated using any suitable modelingtechniques/technologies. In some embodiments, modeling engine 301 maycompare the 3D model 303 with other sensor data (e.g., image or videodata) captured by the sensors 220.

The NTMRM 300 may include code of a user input analysis engine 311 todirect the processor 302 to obtain, from sensors 220 of the sensor array120, second sensor data that is representative of one or more userinteractions. The user interactions may include gestures performed bythe user 108 and/or voice commands issued by the user 108. In someembodiments, the second sensor data may include sensor data obtainedfrom the wearable 110 worn by the user 108, which may be used to augmentor supplement the second sensor data.

With regard to gesture recognition, in one example, the user inputanalysis engine 311 may detect gestures using key pointers in a3D/skeletal-based model or an appearance-based model, including featuredetection of body parts, and may determine actions utilizing a gesturelibrary that correlates a specific gesture to one or more particularactions. In embodiments where the environment 115 includes a capacitive,resistive, or other like touch-surface, the user input analysis engine311 may obtain a touch signal from circuitry of the touch-surface. Thetouch signal may include information regarding a location of the touch(e.g., one or more sets of (x,y) coordinates describing an area, shapeor skeleton of the touch), a pressure of the touch (e.g., as measured byarea of contact between a user's finger or a deformable stylus and thetouchscreen 104, or by a pressure sensor), a duration of contact, anyother suitable information, or any combination of such information. Inthis embodiment, the user input analysis engine 311 may identifygesture(s) based on the information of the touch signal, and utilizingthe gesture library discussed previously.

With regard to voice recognition, the user input analysis engine 311 mayperform speech recognition using acoustic modeling, language modelingapproaches, hidden Markov modeling, dynamic time wrapping, neuralnetworks, and/or the like; and may use natural language processing (NLP)algorithm(s) to identify a particular voice command from the recognizedspeed. The detected voice commands may be translated into one or moreactions using a voice command library, which may be the same or similarto the gesture library discussed previously. The actions may includegenerating and providing various projection/display, haptic, sound,and/or chemical outputs. The actions may also include generating andsending messages to instrumented objects 131I to instruct theinstrumented objects 131I to perform various actions.

The NTMRM 300 may include code of an object recognition engine 321 todirect the processor 302 to identify a physical object within the 3Dmodel 303. For example, the object recognition engine 321 may include afeature detector, a hypothesizer, a hypothesis verifier, and a modeldatabase that includes object models. The feature detector may applyoperators to the 3D model 303 and may identify locations of variousfeatures for forming object hypotheses. A feature may be an attribute ofan object, such as size, color, shape, relation to other objects, etc.The features used by the object recognition engine 321 may depend on theobjects indicated by the object profiles 322 and/or the tracking history323, and the organization of the model database. Using the detectedfeatures, the hypothesizer may assign a likelihood/probability to eachpotential object in the 3D model 303 to produce candidate objects. Theverifier may use object models from the model database to verify thehypotheses and refine the likelihood/probability assigned to theobjects. The object models in the model database may be qualitative orfunctional description, geometric surface information, and/or abstractfeature vectors. The model database may be organized using some type ofindexing scheme to facilitate elimination of unlikely object candidatesfrom possible consideration. The object recognition engine 321 may, foreach candidate object, select an object from an object model with ahighest likelihood/probability as the detected object.

As shown, the object recognition engine 321 may include object profiles322 and tracking history 323. Object profiles 322 may indicate variousattributes for a detected object 131 (including physical objects 131P,virtual objects 131V, instrumented objects 131I, and/or other bindings135), such as a size, shape, color, etc. to be used for the objectrecognition, and may include or indicate various binding orgameplay-related attributes that may influence the creation and/ormanipulation of various bindings 135. The binding or gameplay-relatedattributes may indicate various physical properties that a binding 135should have/exhibit/emulate, for example, size, shape, color(s),elasticity, stiffness, fluidity, etc. In role-player games (RPGs) orfighting or duel-based games, the binding or gameplay-related attributesmay indicate hit points, lifespan, or other like attributes.Additionally, the object profiles 322 may also includerelationship-based attributes, which may be attributes applied to abinding 135 based on an associated object 131. For example, a binding135 may include one or more virtual binding to physical object (VB-PO)attributes when a user 108 associates a binding 135 with one or morephysical objects 131P, one or more virtual binding to VO (VB-VO)attributes when the user 108 associates a binding 135 with one or morevirtual objects 131V, and one or more virtual binding to instrumentedobject 131I (VB-IO) attributes when the user 108 associates a binding135 with one or more instrumented objects 131I. The VB-PO, VB-VO, and/orVB-IO attributes may be based on the attributes of the correspondingphysical object 131P, virtual object 131V, and/or instrumented object131I as indicated by the object profiles 322. Furthermore, the objectprofiles 322 of instrumented objects 131I may include information forinstructing an instrumented object 131I to perform various actions, suchas the instructions to perform individual actions, supported messageformats, supported signaling protocols, and/or the like. The trackinghistory 323 may indicate previously detected objects and/or attributesof the previously detected objects, and may influence the attributesassociated with the bindings 135. The object profiles 322 and trackinghistory 323 may be formed using any suitable format, such as XML, JSON,Abstract Syntax Notation One (ASN.1), ARML, or some other suitable datastructure.

The NTMRM 300 may include code of an AR engine 331 to direct theprocessor 302 to generate an instance of a virtual binding 135 based onone or more identified/detected objects 131 (e.g., as determined by theobject recognition engine 321) and the identified/detected userinteractions (e.g., as determined by the user input analysis engine311). The instance of the virtual binding 135 may have one or moreattributes that influence one or more actions that may be performed bythe virtual binding 135 or using the binding 135, which may be based onthe attributes of an object profile 322 of the detected objects 131. TheAR engine 331 may include binding models 331, which may describe ordefine a process for generating an instance of the binding 135 based onthe one or more attributes of the object profile 322. The AR renderer333 may operate in conjunction with the AR engine 331 to display/projectthe generated images according to one or more known video mapping orprojection mapping methods, one or more known projection methods, and/orany other like known method for displaying/projecting images. Thehardware devices used by the AR engine 331 and the AR renderer 333 mayinclude one or more GPUs and/or any other like hardware device that isconfigured to perform complex image generation and/or renderingcalculations.

The AR engine 331 may provide the generated instance of the binding 135to interface circuitry, which may provide the instance of the virtualbinding 135 to be displayed/rendered/projected such that, upon displayof the instance, the virtual binding 135 may appear to be attached tothe object 131 or appear to interact with the object 131. Additionally,the AR engine 331 may provide the instructions/commands to interfacecircuitry and/or communication circuitry, which may provide theinstructions/commands to instrumented objects 131I to perform variousactions.

In a first example, when a detected user interaction is a userinteraction to indicate a selection of a particular virtual binding 135,the AR engine 331 may obtain one or more object attributes from anobject profile 322 of the detected object 131, and generate a selectioninstance of the virtual binding 135 to emulate various physicalproperties indicated by the one or more obtained attributes.

In a second example, when the detected user interaction is a userinteraction to associate the virtual binding 135 with the detectedobject 131, the AR engine 331 may generate an association instance ofthe virtual binding 135 such that the virtual binding 135 appears to beattached to, or otherwise interact with the detected object 131 upondisplay. Depending upon an object type of the detected object 131, he ARengine 331 may identify one or more VB-PO/VO/IO attributes of theassociation of the virtual binding 135 with the physical object 131P,and the VB-PO/VO/IO attributes may indicate how the virtual binding 135is to interact and/or be manipulated.

In embodiments, the AR engine 331 may also generate instances of virtualobjects 131V based on the detected objects 131, detected userinteractions, and/or various bindings 135. The virtual objects 131V mayinclude object attributes indicated by corresponding object profiles322. The AR engine 331 may provide the generated instance of the virtualobjects 131V to the interface circuitry, which may provide the instanceof the virtual objects 131V to be displayed/rendered/projected.

The NTMRM 300 may include code of a context engine 341 to direct theprocessor 302 to determine one or more contextual attributes, which maybe used to influence how the binding 135 is to be created and/ormanipulated. The context engine 341 may determine aspects such as useractivity, user attributes, semantic location, social circumstances,ambient/environmental conditions, presence of electronic devices,schedules, communication, etc. that may allow the AR platform 105 todetermine what desirable actions may be in a given circumstance. Theuser activity contexts may include any information related to user 108,such as body (or body part) positions/orientations. The user attributesmay include any information related to user 108, such as user'spreferences for various games or objects 131, and/or any other likepreferences indicating a user's tastes, interests, goals, and the like.The user preferences may be set by the user 108, or obtained from one ormore applications 351 running on the computing devices 110. The userpreferences may be obtained explicitly (e.g., by obtaining user ratingsand/or rankings) or implicitly (e.g., by mining click-through data, logdata, user profile data, and the like of the content/apps 351). The userattributes may also include other information such as an internetservice or cellular service subscription type or data plan, asubscription to one or more content providing services, userpreferences, demographic information, etc. The environmental-basedcontextual information may include information, such as a distancebetween the user 108 and various objects 131 in the environment 115,ambient lighting, background noise, surrounding electromagnetic fields,and/or any other like biotic or abiotic factors surrounding the user 108and/or environment 115. The contexts may also include device attributes,such as information indicative of one or more peripheral devices (e.g.,sensors 220, EMCs 222, and the like) and/or internal components (e.g.,processor types, memory devices, etc.) of the AR platform 105 and/orcomponents of the environment 115.

Although FIG. 3 shows program code of various modules or engines storedby the NTMRM 300, in some implementations, a combination of the programcode with the hardware elements (or a combination of circuits used in anelectrical or electronic system) used to carry out the functionality ofthat program code (e.g., processor 302, NTMRM 300, bus 306, and thelike) may be referred to as a particular type of circuitry. For example,a combination of various hardware elements and the modeling engine 301may be referred to as “modeling circuitry”, a combination of varioushardware elements and the user input analysis engine 311 may be referredto as “user input circuitry”, a combination of various hardware elementsand the object recognition engine 321 may be referred to as “objectrecognition circuitry”, a combination of various hardware elements andthe AR engine 331 may be referred to as “AR circuitry”, and acombination of various hardware elements and the context engine 341 maybe referred to as “context circuitry”. Other combinations may bepossible in other embodiments.

FIGS. 4-5 illustrates processes 400-500 for generating and displayingbindings 135 in accordance with various embodiments. For illustrativepurposes, the operations of the aforementioned processes are describedas being performed by the various elements discussed with regard toFIGS. 1-3. However, it should be noted that other computing devices (orcomponents thereof) may operate the processes 400-500 in a multitude ofimplementations, arrangements, and/or environments. In embodiments, theprocesses 400-500 may be implemented as program code, which whenexecuted by a processor, causes a computer system to perform the variousoperations of the processes. While particular examples and orders ofoperations are illustrated in FIGS. 4-5, in various embodiments, theseoperations may be re-ordered, separated into additional operations,combined, or omitted altogether.

Referring to FIG. 4, process 400 may begin after the AR platform 105 isturned on or otherwise activated, or when the user 108 controls the ARplatform 105 to being operation. At operation 405, a user input analysisengine 311 of the AR platform 105 may identify a selection of a virtualbinding 135, which may be based on sensor data that is representative ofone or more user interactions (e.g., gestures, voice commands, etc.). Atoperation 410, the user input analysis engine 311 may detect a userinteraction to attach the virtual binding 135 to a physical object 131P.The AR platform 105 may control the sensors 220 of the sensor array 120to scan the environment 115, and may generate sensor data that isrepresentative of the scanned environment 115. The AR system 115 mayobtain this sensor data, and generate a 3D model 303 of the environment115, including any physical or instrumented objects 131 that are in theenvironment 115. After the 3D model 303 is created, the AR platform 105may control the sensors 220 of the sensor array 120 to detect userinteractions, including gestures and/or voice commands.

At operation 415, the object recognition engine 321 of the AR platform105 may identify the physical object 131P, and at operation 420, the ARengine 331 of the AR platform 105 may determine one or more physicalattributes of the physical object 131P using an object profile 322 ofthe identified physical object 131P. At operation 420, the AR engine 331of the AR platform 105 may determine virtual binding 135 to physicalobject 131P (VB-PO) attributes based on the physical object 131Pattributes and/or using the object profile 322. At operation 430, ARengine 331 of the AR platform 105 may generate an instance of thevirtual binding 135, and may control the interface circuitry of the ARplatform 105 to provide the instance of the virtual binding 135 fordisplay.

At operation 435, the user input analysis engine 311 of the AR platform105 may detect another user interaction to associate a virtual object131V with the virtual binding 135. At operation 440, the AR engine 331of the AR platform 105 may determine virtual binding 135 to virtualobject 131V (VB-VO) attributes, which may be based on attributes of thevirtual object 131V and/or an object profile 322 of the virtual object131V. At operation 445, the user input analysis engine 311 of the ARplatform 105 may detect a user interaction to manipulate the virtualbinding 135. At operation 445, AR engine 331 of the AR platform 105 maygenerate a manipulate instance of the virtual binding 135, and maycontrol the interface circuitry of the AR platform 105 to provide themanipulate instance of the virtual binding 135 to be display. Afterperforming operation 450, process 400 may repeat as necessary or end.

Referring to FIG. 5, process 500 may begin after the AR platform 105 isturned on or otherwise activated, or when the user 108 controls the ARplatform 105 to being operation. At operation 505, a user input analysisengine 311 of the AR platform 105 may identify a selection of a virtualbinding 135, which may be based on sensor data that is representative ofone or more user interactions (e.g., gestures, voice commands, etc.). Atoperation 510, the user input analysis engine 311 may detect a userinteraction to attach the virtual binding 135 to an instrumented object131V. Operations 505 and 510 may be the same or similar to operations405 and 410 of process 400.

At operation 515, the object recognition engine 321 of the AR platform105 may identify the instrumented object 131I. At operation 520, the ARengine 331 of the AR platform 105 may control the communicationcircuitry 205 to establish a connection with the instrumented object131I. At operation 525, the AR engine 331 of the AR platform 105 maygenerate an instance of the virtual binding 135, and may controlinterface circuitry of the AR platform 105 to provide the instance ofthe virtual binding 135 to be display.

At operation 530, the AR engine 331 of the AR platform 105 may determineattributes of the instrumented object 131I and/or VB-IO attributes basedon the instrumented object 131I attributes. These attributes may beidentified from an object profile 322 associated with the instrumentedobject 131I, which is stored by the object recognition engine 321. Theseattributes may also include instructions and/or other like informationfor instructing the instrumented object 131I to perform one or moreactions.

At operation 535, the user input analysis engine 311 may detect a userinteraction to manipulate the virtual binding 135 and/or theinstrumented object 131I, and at operation 540, the AR engine 331 of theAR platform 105 may control the communication circuitry 205 to transmitselected instructions to the instrumented object 131I to perform one ormore actions based on the user interaction detected at operation 535.After performance of operation 540, process 500 may repeat as necessaryor end.

FIGS. 6-7 illustrate various user interaction stages in accordance withvarious embodiments. The example shown by FIGS. 6-7 may be aslingshot-bowling game, where a user attempts to knock down a set ofpins (e.g., virtual object 630V-1) using a projectile (virtual object630V-2) using a virtual binding (e.g., binding 635).

Referring to FIG. 6, at stage 1 the user 608 (represented by the hand inFIG. 6) may cause a binding 635 to be generated by the AR platform 105to bind two physical objects 630P-1 and 630P-2 by performing a tap-holdgesture using an index finger at a first physical object 630P-1, anddragging the tip of the index finger from the first physical object630P-1 towards a second physical object 630P-2. Once the index finger ofthe user 608 reaches the second physical object 630P-2, the AR platform105 may connect an end of the binding 635 to the second physical object630P-2, and the user 608 may release the hold gesture. In this example,the physical objects 630P-1 and 630P-2 may represent the tips of aY-frame typically used in classic slingshots. Stage 1 also shows a thirdphysical object 630P-3 (e.g., a ruler) and a projected virtual object630V-1 (e.g., a pin formation in FIG. 5). In this example, the game mayrequire the user 608 to shoot a projectile to knock down pins of object630V-1 by ricocheting off of the object 630P-3.

At stage 2, the user 608 may perform a grab and hold gesture on thebinding and then perform a pulling gesture to stretch the binding 635towards the user 608. As the user 608 stretches the binding 635,multiple instances of the binding 635 may be generated by the ARplatform 105, and displayed in such a way that the binding 635 appearsto exhibit various physical properties. For example, typical slingshotsoperate using stored elastic energy to shoot a projectile at high speed,where the elastic energy is created and stored as a slingshot operatordraws back rubber bands that are connected to a Y-frame of theslingshot. In this example, the AR platform 105 may emulate elasticenergy to be stored in the binding 635 as the user 608 performs the graband pull gestures, such as by using linear or non-linear elasticityequations.

At stage 3, a virtual object 630V-2 may be generated by the AR platform105, and displayed. In this example, the virtual object 630V-2 may be aprojectile to be shot at the virtual object 630V-1 via physical object630P-3. In embodiments, the gestures performed with or around thebinding 635 may indicate a desired virtual object to be generated. Inthis example, the act of pulling the binding 635 towards the user 608 atstage 2 may indicate that the user 608 wishes to create a slingshotpocket. In response to detecting the grab and/or pulling gestures atstage 2, the AR platform 105 may generate and display the projectileobject 630V-2 in a slingshot pocket created by the bent binding 635 atstage 3.

Referring to FIG. 7, at stage 4 the user 608 may perform a gesture torelease the binding 635, and in response, the AR platform 105 maygenerate and display various images or an animation showing the binding635 and virtual object 630V-2 reacting to the gesture, which is shown bystages 4, 5, and 6. In this example, upon performing the releasegesture, various instances of the binding 635 may be generated anddisplayed showing the elastic energy stored by the binding 635 beingreleased and translated into kinetic energy of the projectile object630V-2. At stage 6, the AR platform 105 may generate and display variousimages or an animation showing the virtual object 630V-2 ricocheting offof the physical object 630P-3. At stage 7, the AR platform 105 maygenerate and display various images or an animation showing the virtualobject 630V-2 and the virtual object 630V-1 interacting with oneanother. In this example, the AR platform 105 may generate and displayvarious images or an animation of set of pins 630V-3 being knocked downby the projectile object 630V-2 after it ricocheted off physical object630P-3.

FIG. 8 illustrates various user interaction stages in accordance withvarious embodiments. The example shown by FIG. 8 shows a user 808interacting with an instrumented object 830I. In this example, theinstrumented object 830I is a toy car, which may have circuitry tocommunicate with the AR platform 105 and may also have circuitry (e.g.,processors, memory, etc.) to control EMCs, such as a wheels to propelthe toy car. Additionally, the binding 835 may act as, or emulate, arope, string, rubber band, handle, or some other like connector.

At stage 1 the user 808 (represented by the hand and arm in FIG. 8) maycause a binding 835 to be generated by the AR platform 105 to bound withinstrumented object 830I by performing a tap-hold gesture using an indexfinger at the instrumented object 830I, and dragging the tip of theindex finger away from the instrumented object 830I. Once the indexfinger of the user 808 reaches a predetermined distance away from theinstrumented object 830I or when the user 808 stops dragging the indexfinger, the AR platform 105 may set or establish the binding 835.

At stage 2, the user 808 may perform a pinch and hold gesture toindicate to interact or manipulate the binding 835 and/or instrumentedobject 830I. At stages 2-3, the user 808 may perform a dragging gesture,while continuing to perform the pinch and hold gesture, to move theinstrumented object 830I using the binding 835. In stages 2-3, the ARplatform 105 may send instructions to the instrumented object 830I thatinstructs the instrumented object 830I to take one or more actions whilethe user 808 performs the dragging gesture. In this example, theinstructions may instruct the instrumented object 830I to activate itsEMCs in order to propel the instrumented object 830I in a direction thatcorresponds with the dragging gesture.

Some non-limiting examples are as follows.

Example 1 may include a computer device employed in an augmented reality(AR) system, the computer device comprising: modeling circuitry toobtain first sensor data from a sensor array, wherein the first sensordata is representative of a physical environment and physical objects inthe physical environment, and to track the physical objects in thephysical environment based on the first sensor data; user inputcircuitry to obtain, from the sensor array, second sensor data that isrepresentative of a gesture performed by a user; object recognitioncircuitry to identify a tracked physical object; AR circuitry togenerate an instance of a virtual binding based on the identifiedphysical object and the gesture, wherein the instance of the virtualbinding has one or more virtual binding attributes that influence one ormore actions to be performed by the virtual binding based on theidentified physical object; and interface circuitry to provide theinstance of the virtual binding to be displayed such that, upon displayof the instance, the virtual binding is to appear to be attached to thephysical object or appear to interact with the physical object.

Example 2 may include the computer device of example 1 and/or some otherexample herein, wherein the AR circuitry is to generate the instance ofthe virtual binding when the gesture is a gesture to indicate aselection of a particular virtual binding.

Example 3 may include the computer device of example 1 and/or some otherexample herein, wherein the AR circuitry is to obtain one or more objectattributes of the physical object, and generate the instance of thevirtual binding based on the one or more object attributes.

Example 4 may include the computer device of example 1 and/or some otherexample herein, wherein the instance is a selection instance, andwherein: the AR circuitry is to: generate an association instance of thevirtual binding when the gesture is a gesture to indicate to associatethe virtual binding with the physical object, and identify one or morevirtual binding to physical object (VB-PO) attributes of the associationof the virtual binding with the physical object; and the interfacecircuitry to provide the association instance of the virtual binding tobe displayed, wherein the one or more VB-PO attributes are to indicatewhether, upon display of the association instance, the virtual bindingis to appear to be attached to the physical object or appear to interactwith the physical object.

Example 5 may include the computer device of example 4 and/or some otherexample herein, wherein: the AR circuitry is to generate a PO-manipulateinstance of the virtual binding when the gesture is a gesture tomanipulate the virtual binding; and the interface circuitry to providethe PO-manipulate instance of the virtual binding to be displayed suchthat the virtual binding is to appear to be manipulated in response toperformance of the gesture.

Example 6 may include the computer device of example 5 and/or some otherexample herein, further comprising: context circuitry to determine oneor more contextual attributes, wherein the one or more contextualattributes comprise user activity, semantic location, socialcircumstances, ambient conditions, presence of one or more electronicdevices, schedules, and user communications, wherein the AR circuitry isto generate the PO-manipulate instance based on the one or morecontextual attributes.

Example 7 may include the computer device of examples 1-6 and/or someother example herein, wherein the instance is a selection instance, andwherein: the AR circuitry is to: generate a virtual object (VO) when thegesture is a gesture to indicate to select the VO, generate a VOinstance of the virtual binding when the gesture is a gesture toindicate to associate the virtual binding with the VO, identify one ormore virtual binding to VO (VB-VO) attributes of the association of thevirtual binding with the VO, and generate a VO-manipulate instance ofthe virtual binding when the gesture is a gesture to manipulate thevirtual binding; and the interface circuitry to: provide the VO and theVO instance of the virtual binding to be displayed, wherein the one ormore VB-VO attributes are to indicate whether, upon display of the VOinstance and the VO, the virtual binding is to appear to be attached tothe VO or appear to interact with the VO, and provide the VO-manipulateinstance of the virtual binding to be displayed such that the virtualbinding is to appear to be manipulated in response to performance of thegesture to manipulate the virtual binding.

Example 8 may include the computer device of examples 1-7 and/or someother example herein, wherein the modeling circuitry is to generate athree-dimensional (3D) model of the physical environment, and whereinthe object recognition circuitry to identify the physical object withinthe 3D model.

Example 9 may include the computer device of examples 1-8 and/or someother example herein, wherein the instance is a selection instance, andwherein: the AR circuitry is to: generate a instrumented object (IO)instance of the virtual binding when the gesture is a gesture toindicate to associate the virtual binding with an instrumented object,and identify one or more virtual binding to IO (VB-IO) attributes of theassociation of the virtual binding with the VO; and the interfacecircuitry to: cause a network connection to be established with theinstrumented object, and provide the IO instance of the virtual bindingto be displayed, wherein the one or more VB-IO attributes are toindicate whether, upon display of the IO instance, the virtual bindingis to appear to be attached to the instrumented object or appear tointeract with the instrumented object.

Example 10 may include the computer device of example 9 and/or someother example herein, wherein: the AR circuitry is to generate aIO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding to act on the instrumentedobject; and the interface circuitry to: provide the IO-manipulateinstance of the virtual binding to be displayed such that the virtualbinding is to appear to be manipulated in response to performance of thegesture, and cause a message to be transmitted to the instrumentedobject over the network connection, wherein the instruction is toinstruct the instrumented object to perform one or more actions inresponse to the manipulation of the virtual binding.

Example 11 may include one or more computer-readable media (CRM)comprising instructions, which when executed by one or more processorsof a computer device, is to cause the computer device to: obtain firstsensor data from a sensor array, wherein the first sensor data isrepresentative of a physical environment; track physical objects in thephysical environment based on the first sensor data; identify an objectand object attributes associated with the identified object; obtain,from the sensor array, second sensor data that is representative of agesture performed by a user; identify the gesture based on the secondsensor data; generate, in response to identification of the gesture, aninstance of a virtual binding based on the object attributes, whereinthe instance of the virtual binding has one or more virtual bindingattributes that influence one or more actions to be performed by thevirtual binding; and provide the instance of the virtual binding to bedisplayed such that, upon display of the instance, the virtual bindingis to appear to be attached to the object or appear to interact with theobject.

Example 12 may include the one or more CRM of example 11 and/or someother example herein, wherein the instance of the virtual binding is aselection instance when the gesture is a gesture to indicate a selectionof a particular virtual binding.

Example 13 may include the one or more CRM of example 11 and/or someother example herein, wherein execution of the instructions is to causethe computer device to obtain an object profile associated with theidentified object, identify the object attributes from the objectprofile, and determine the virtual binding attributes based on theobject attributes.

Example 14 may include the one or more CRM of example 11 and/or someother example herein, wherein the object is a physical object, andexecution of the instructions is to cause the computer device to:generate an association instance of the virtual binding when the gestureis a gesture to indicate to associate the virtual binding with thephysical object; identify one or more virtual binding to physical object(VB-PO) attributes of the association of the virtual binding with thephysical object; and provide the association instance of the virtualbinding to be displayed, wherein the one or more VB-PO attributes are toindicate whether, upon display of the association instance, the virtualbinding is to appear to be attached to the physical object or appear tointeract with the physical object.

Example 15 may include the one or more CRM of example 14 and/or someother example herein, wherein execution of the instructions is to causethe computer device to: generate a PO-manipulate instance of the virtualbinding when the gesture is a gesture to manipulate the virtual binding;and provide the PO-manipulate instance of the virtual binding to bedisplayed such that the virtual binding is to appear to be manipulatedin response to performance of the gesture and according to the virtualbinding attributes.

Example 16 may include the one or more CRM of example 15 and/or someother example herein, wherein execution of the instructions is to causethe computer device to: determine one or more contextual attributes,wherein the one or more contextual attributes comprise user activity,semantic location, social circumstances, ambient conditions, presence ofone or more electronic devices, schedules, and user communications; andgenerate the PO-manipulate instance based on the one or more contextualattributes.

Example 17 may include the one or more CRM of example 11 and/or someother example herein, wherein execution of the instructions is to causethe computer device to: generate a virtual object (VO) when the gestureis a gesture to indicate to select the VO; generate a VO instance of thevirtual binding when the gesture is a gesture to indicate to associatethe virtual binding with the VO; identify one or more virtual binding toVO (VB-VO) attributes of the association of the virtual binding with theVO; generate a VO-manipulate instance of the virtual binding when thegesture is a gesture to manipulate the virtual binding; provide the VOand the VO instance of the virtual binding to be displayed, wherein theone or more VB-VO attributes are to indicate whether, upon display ofthe VO instance and the VO, the virtual binding is to appear to beattached to the VO or appear to interact with the VO and in accordancewith the virtual binding attributes; and provide the VO-manipulateinstance of the virtual binding to be displayed such that the virtualbinding is to appear to be manipulated in response to performance of thegesture and according to the virtual binding attributes.

Example 18 may include the one or more CRM of example 17 and/or someother example herein, wherein execution of the instructions is to causethe computer device to: generate a three-dimensional (3D) model of thephysical environment for an AR environment based on the sensor data; andidentify the physical object within the 3D model.

Example 19 may include the one or more CRM of example 11 and/or someother example herein, wherein execution of the instructions is to causethe computer device to: generate a instrumented object (IO) instance ofthe virtual binding when the gesture is a gesture to indicate toassociate the virtual binding with an instrumented object; identify oneor more virtual binding to IO (VB-IO) attributes of the association ofthe virtual binding with the VO; cause a network connection to beestablished with the instrumented object; and provide the IO instance ofthe virtual binding to be displayed, wherein the one or more VB-IOattributes are to indicate whether, upon display of the IO instance, thevirtual binding is to appear to be attached to the instrumented objector appear to interact with the instrumented object and according to thevirtual binding attributes.

Example 20 may include the one or more CRM of example 19 and/or someother example herein, wherein execution of the instructions is to causethe computer device to: generate a IO-manipulate instance of the virtualbinding when the gesture is a gesture to manipulate the virtual bindingto act on the instrumented object; provide the IO-manipulate instance ofthe virtual binding to be displayed such that the virtual binding is toappear to be manipulated in response to performance of the gesture; andcause a message to be transmitted to the instrumented object over thenetwork connection, wherein the instruction is to instruct theinstrumented object to perform one or more actions in response to themanipulation of the virtual binding.

Example 21 may include a method to be performed by a computer device,the method comprising: obtaining, by the computer device, first sensordata from a sensor array, wherein the first sensor data isrepresentative of a physical environment; tracking, by the computerdevice, physical objects in the physical environment based on the firstsensor data; identifying, by the computer device, an object within thephysical environment and object attributes associated with theidentified object; obtain, by the computer device from the sensor array,second sensor data that is representative of a gesture performed by auser; identifying, by the computer device, the gesture based on thesecond sensor data; generating, by the computer device in response toidentification of the gesture, an instance of a virtual binding based onthe object attributes, wherein the instance of the virtual binding hasone or more virtual binding attributes that influence one or moreactions to be performed by the virtual binding; and providing, by thecomputer device, the instance of the virtual binding to be displayedsuch that, upon display of the instance, the virtual binding is toappear to be attached to the object or appear to interact with theobject.

Example 22 may include the method of example 21 and/or some otherexample herein, wherein the instance of the virtual binding is aselection instance when the gesture is a gesture to indicate a selectionof a particular virtual binding.

Example 23 may include the method of example 21 and/or some otherexample herein, further comprising: obtaining, by the computer device,an object profile associated with the identified object; identifying, bythe computer device, the object attributes from the object profile; anddetermining, by the computer device, the virtual binding attributesbased on the object attributes.

Example 24 may include the method of example 21 and/or some otherexample herein, wherein the object is a physical object, and executionof the instructions is to cause the computer device to: generating, bythe computer device, an association instance of the virtual binding whenthe gesture is a gesture to indicate to associate the virtual bindingwith the physical object; identifying, by the computer device, one ormore virtual binding to physical object (VB-PO) attributes of theassociation of the virtual binding with the physical object; andproviding, by the computer device, the association instance of thevirtual binding to be displayed, wherein the one or more VB-POattributes are to indicate whether, upon display of the associationinstance, the virtual binding is to appear to be attached to thephysical object or appear to interact with the physical object.

Example 25 may include the method of example 24 and/or some otherexample herein, further comprising: generating, by the computer device,a PO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding; and providing, by thecomputer device, the PO-manipulate instance of the virtual binding to bedisplayed such that the virtual binding is to appear to be manipulatedin response to performance of the gesture and according to the virtualbinding attributes.

Example 26 may include the method of example 25 and/or some otherexample herein, wherein execution of the instructions is to cause thecomputer device to: determining, by the computer device, one or morecontextual attributes, wherein the one or more contextual attributescomprise user activity, semantic location, social circumstances, ambientconditions, presence of one or more electronic devices, schedules, anduser communications; and generating, by the computer device, thePO-manipulate instance based on the one or more contextual attributes.

Example 27 may include the method of example 21 and/or some otherexample herein, further comprising: generating, by the computer device,a virtual object (VO) when the gesture is a gesture to indicate toselect the VO; generating, by the computer device, a VO instance of thevirtual binding when the gesture is a gesture to indicate to associatethe virtual binding with the VO; identifying, by the computer device,one or more virtual binding to VO (VB-VO) attributes of the associationof the virtual binding with the VO; generating, by the computer device,a VO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding; providing, by the computerdevice, the VO and the VO instance of the virtual binding to bedisplayed, wherein the one or more VB-VO attributes are to indicatewhether, upon display of the VO instance and the VO, the virtual bindingis to appear to be attached to the VO or appear to interact with the VOand in accordance with the virtual binding attributes; and providing, bythe computer device, the VO-manipulate instance of the virtual bindingto be displayed such that the virtual binding is to appear to bemanipulated in response to performance of the gesture and according tothe virtual binding attributes.

Example 28 may include the method of example 27 and/or some otherexample herein, wherein execution of the instructions is to cause thecomputer device to: generating, by the computer device, athree-dimensional (3D) model of the physical environment for an ARenvironment based on the sensor data; and identifying, by the computerdevice, an object within the 3D model.

Example 29 may include the method of example 21 and/or some otherexample herein, wherein the instance is a first instance, and whereinexecution of the instructions is to cause the computer deviceto:generating, by the computer device, a instrumented object (IO)instance of the virtual binding when the gesture is a gesture toindicate to associate the virtual binding with an instrumented object;identifying, by the computer device, one or more virtual binding to IO(VB-IO) attributes of the association of the virtual binding with theVO; establishing, by the computer device, a network connection with theinstrumented object; and providing, by the computer device, the IOinstance of the virtual binding to be displayed, wherein the one or moreVB-IO attributes are to indicate whether, upon display of the IOinstance, the virtual binding is to appear to be attached to theinstrumented object or appear to interact with the instrumented objectand according to the virtual binding attributes.

Example 30 may include the method of example 29 and/or some otherexample herein, wherein execution of the instructions is to cause thecomputer device to: generating, by the computer device, a IO-manipulateinstance of the virtual binding when the gesture is a gesture tomanipulate the virtual binding to act on the instrumented object;providing, by the computer device, the IO-manipulate instance of thevirtual binding to be displayed such that the virtual binding is toappear to be manipulated in response to performance of the gesture; andtransmitting, by the computer device, a message to the instrumentedobject over the network connection, wherein the instruction is toinstruct the instrumented object to perform one or more actions inresponse to the manipulation of the virtual binding.

Example 31 may include an augmented reality (AR) system, comprising: anenvironment comprising a sensor array and an output device array,wherein the sensor array comprises one or more sensors and the outputdevice array comprises one or more electromechanical devices; and an ARplatform communicatively coupled with the sensor array and the outputdevice array, the AR platform comprising: processor circuitry coupledwith memory circuitry, communication circuitry, and interface circuitry,wherein the processor circuitry is to: control the interface circuitryor the communication circuitry to obtain first sensor data from the oneor more sensors of the sensor array; track physical objects within theenvironment based on the first sensor data; identify an object withinthe 3D model and object attributes associated with the identifiedobject; obtain second sensor data from the one or more sensors of thesensor array, wherein the second sensor data is representative of agesture performed by a user; identify the gesture based on the secondsensor data; generate, in response to identification of the gesture, aninstance of a virtual binding based on the object attributes, whereinthe instance of the virtual binding has one or more virtual bindingattributes that influence one or more actions to be performed by thevirtual binding; and control the interface circuitry or thecommunication circuitry to provide the instance of the virtual bindingto an individual output device of the one or more output devices of theoutput device array, wherein the individual output device is todisplayed the instance of the virtual binding such that, upon display ofthe instance, the virtual binding is to appear to be attached to theobject or appear to interact with the object.

Example 32 may include the AR system of example 31 and/or some otherexample herein, wherein the instance of the virtual binding is aselection instance when the gesture is a gesture to indicate a selectionof a particular virtual binding.

Example 33 may include the AR system of example 31 and/or some otherexample herein, wherein the processor circuitry is to obtain an objectprofile associated with the identified object, identify the objectattributes from the object profile, and determine the virtual bindingattributes based on the object attributes.

Example 34 may include the AR system of example 31 and/or some otherexample herein, wherein the processor circuitry is to: generate anassociation instance of the virtual binding when the gesture is agesture to indicate to associate the virtual binding with the physicalobject; identify one or more virtual binding to physical object (VB-PO)attributes of the association of the virtual binding with the physicalobject; and control the interface circuitry or the communicationcircuitry to provide the association instance of the virtual binding tobe displayed, wherein the one or more VB-PO attributes are to indicatewhether, upon display of the association instance, the virtual bindingis to appear to be attached to the physical object or appear to interactwith the physical object.

Example 35 may include the AR system of example 34 and/or some otherexample herein, wherein the processor circuitry is to: generate aPO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding; and control the interfacecircuitry or the communication circuitry to provide the PO-manipulateinstance of the virtual binding to be displayed such that the virtualbinding is to appear to be manipulated in response to performance of thegesture and according to the virtual binding attributes.

Example 36 may include the AR system of example 35 and/or some otherexample herein, wherein the processor circuitry is to: determine one ormore contextual attributes, wherein the one or more contextualattributes comprise user activity, semantic location, socialcircumstances, ambient conditions, presence of one or more electronicdevices, schedules, and user communications; and generate thePO-manipulate instance based on the one or more contextual attributes.

Example 37 may include the AR system of example 31 and/or some otherexample herein, the processor circuitry is to: generate a virtual object(VO) when the gesture is a gesture to indicate to select the VO;generate a VO instance of the virtual binding when the gesture is agesture to indicate to associate the virtual binding with the VO;identify one or more virtual binding to VO (VB-VO) attributes of theassociation of the virtual binding with the VO; generate a VO-manipulateinstance of the virtual binding when the gesture is a gesture tomanipulate the virtual binding; control the interface circuitry or thecommunication circuitry to provide the VO and the VO instance of thevirtual binding to be displayed, wherein the one or more VB-VOattributes are to indicate whether, upon display of the VO instance andthe VO, the virtual binding is to appear to be attached to the VO orappear to interact with the VO and in accordance with the virtualbinding attributes; and control the interface circuitry or thecommunication circuitry to provide the VO-manipulate instance of thevirtual binding to be displayed such that the virtual binding is toappear to be manipulated in response to performance of the gesture andaccording to the virtual binding attributes.

Example 38 may include the AR system of example 37 and/or some otherexample herein, the processor circuitry is to: generate athree-dimensional (3D) model of the physical environment for an ARenvironment based on the sensor data; and identify the physical objectwithin the 3D model.

Example 39 may include the AR system of example 31 and/or some otherexample herein, the processor circuitry is to: generate a instrumentedobject (IO) instance of the virtual binding when the gesture is agesture to indicate to associate the virtual binding with aninstrumented object; identify one or more virtual binding to IO (VB-IO)attributes of the association of the virtual binding with the VO; causea network connection to be established with the instrumented object; andcontrol the interface circuitry or the communication circuitry toprovide the IO instance of the virtual binding to be displayed, whereinthe one or more VB-IO attributes are to indicate whether, upon displayof the IO instance, the virtual binding is to appear to be attached tothe instrumented object or appear to interact with the instrumentedobject and according to the virtual binding attributes.

Example 40 may include the AR system of example 39 and/or some otherexample herein, the processor circuitry is to: generate a IO-manipulateinstance of the virtual binding when the gesture is a gesture tomanipulate the virtual binding to act on the instrumented object;control the interface circuitry or the communication circuitry toprovide the IO-manipulate instance of the virtual binding to bedisplayed such that the virtual binding is to appear to be manipulatedin response to performance of the gesture; and control the communicationcircuitry to transmit a message to the instrumented object over thenetwork connection, wherein the instruction is to instruct theinstrumented object to perform one or more actions in response to themanipulation of the virtual binding.

Example 41 may include a computer device employed in an augmentedreality (AR) system, the computer device comprising: modeling means forgenerating a three-dimensional (3D) model of a physical environmentbased on the first sensor data, wherein the first sensor data isobtained from a sensor array; user input analysis means for determininguser interactions based on second senor data obtained from the sensorarray, second sensor data that is representative of user interactions ofa user; object recognition means for identifying a physical objectwithin the 3D model; AR means for generating an instance of a virtualbinding based on the identified physical object and the gesture, whereinthe instance of the virtual binding has one or more virtual bindingattributes that influence one or more actions to be performed by thevirtual binding based on the identified physical object; and displaymeans for displaying the instance of the virtual binding to be displayedsuch that, upon display of the instance, the virtual binding is toappear to be attached to the physical object or appear to interact withthe physical object.

Example 42 may include the computer device of example 41 and/or someother example herein, wherein the AR means is for generating theinstance of the virtual binding when the gesture is a gesture toindicate a selection of a particular virtual binding.

Example 43 may include the computer device of example 41 and/or someother example herein, wherein the AR means is for obtaining one or moreobject attributes of the physical object, and for generating theinstance of the virtual binding based on the one or more objectattributes.

Example 44 may include the computer device of example 41 and/or someother example herein, wherein: the AR means is for generating anassociation instance of the virtual binding when the gesture is agesture to indicate to associate the virtual binding with the physicalobject, and identifying one or more virtual binding to physical object(VB-PO) attributes of the association of the virtual binding with thephysical object; and the display means is for displaying the associationinstance of the virtual binding to be displayed, wherein the one or moreVB-PO attributes are to indicate whether, upon display of theassociation instance, the virtual binding is to appear to be attached tothe physical object or appear to interact with the physical object.

Example 45 may include the computer device of example 44 and/or someother example herein, wherein: the AR means is for generating aPO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding; and the display means is fordisplaying the PO-manipulate instance of the virtual binding to bedisplayed such that the virtual binding is to appear to be manipulatedin response to performance of the gesture.

Example 46 may include the computer device of example 45 and/or someother example herein, further comprising: context means for determiningone or more contextual attributes, wherein the one or more contextualattributes comprise user activity, semantic location, socialcircumstances, ambient conditions, presence of one or more electronicdevices, schedules, and user communications, wherein the AR means is forgenerating the PO-manipulate instance based on the one or morecontextual attributes.

Example 47 may include the computer device of example 41 and/or someother example herein, wherein the instance is a selection instance, andwherein: the AR means is for generating a virtual object (VO) when thegesture is a gesture to indicate to select the VO, generating a VOinstance of the virtual binding when the gesture is a gesture toindicate to associate the virtual binding with the VO, and identifyingone or more virtual binding to VO (VB-VO) attributes of the associationof the virtual binding with the VO; and the display means is fordisplaying the VO and the VO instance of the virtual binding to bedisplayed, wherein the one or more VB-VO attributes are to indicatewhether, upon display of the VO instance and the VO, the virtual bindingis to appear to be attached to the VO or appear to interact with the VO.

Example 48 may include the computer device of example 47 and/or someother example herein, wherein: the AR means is for generating aVO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding; and the display means is fordisplaying the VO-manipulate instance of the virtual binding to bedisplayed such that the virtual binding is to appear to be manipulatedin response to performance of the gesture.

Example 49 may include the computer device of example 41 and/or someother example herein, wherein the instance is a selection instance, andwherein: the AR means is for generating a instrumented object (IO)instance of the virtual binding when the gesture is a gesture toindicate to associate the virtual binding with an instrumented object,and identifying one or more virtual binding to IO (VB-IO) attributes ofthe association of the virtual binding with the VO; and the displaymeans is for displaying the IO instance of the virtual binding to bedisplayed, wherein the one or more VB-IO attributes are to indicatewhether, upon display of the IO instance, the virtual binding is toappear to be attached to the instrumented object or appear to interactwith the instrumented object, and the apparatus comprises communicationmeans for establishing a network connection to be established with theinstrumented object.

Example 50 may include the computer device of example 49 and/or someother example herein, wherein: the AR means is for generating aIO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding to act on the instrumentedobject; and the display means is for displaying the IO-manipulateinstance of the virtual binding to be displayed such that the virtualbinding is to appear to be manipulated in response to performance of thegesture, and the communication means is for transmitting a message tothe instrumented object over the network connection, wherein theinstruction is to instruct the instrumented object to perform one ormore actions in response to the manipulation of the virtual binding.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the disclosed embodiments ofthe disclosed device and associated methods without departing from thespirit or scope of the disclosure. Thus, it is intended that the presentdisclosure covers the modifications and variations of the embodimentsdisclosed above provided that the modifications and variations comewithin the scope of any claims and their equivalents.

1. A computer device employed in an augmented reality (AR) platform, thecomputer device comprising: modeling circuitry to obtain first sensordata from a sensor array, wherein the first sensor data isrepresentative of a physical environment and physical objects in thephysical environment, and to track the physical objects in the physicalenvironment based on the first sensor data; user input circuitry toobtain, from the sensor array, second sensor data that is representativeof a gesture performed by a user; object recognition circuitry toidentify a tracked physical object; AR circuitry to generate an instanceof a virtual binding based on the identified physical object and thegesture, wherein the instance of the virtual binding has one or morevirtual binding attributes that influence one or more actions to beperformed by the virtual binding based on the identified physicalobject; and interface circuitry to provide the instance of the virtualbinding to be displayed such that, upon display of the instance, thevirtual binding is to appear to be attached to the physical object orappear to interact with the physical object.
 2. The computer device ofclaim 1, wherein the AR circuitry is to generate the instance of thevirtual binding when the gesture is a gesture to indicate a selection ofa particular virtual binding.
 3. The computer device of claim 1, whereinthe AR circuitry is to obtain one or more object attributes of thephysical object, and generate the instance of the virtual binding basedon the one or more object attributes.
 4. The computer device of claim 1,wherein the instance is a selection instance, and wherein: the ARcircuitry is to: generate an association instance of the virtual bindingwhen the gesture is a gesture to indicate to associate the virtualbinding with the physical object, and identify one or more virtualbinding to physical object (VB-PO) attributes of the association of thevirtual binding with the physical object; and the interface circuitry toprovide the association instance of the virtual binding to be displayed,wherein the one or more VB-PO attributes are to indicate whether, upondisplay of the association instance, the virtual binding is to appear tobe attached to the physical object or appear to interact with thephysical object.
 5. The computer device of claim 4, wherein: the ARcircuitry is to generate a PO-manipulate instance of the virtual bindingwhen the gesture is a gesture to manipulate the virtual binding; and theinterface circuitry to provide the PO-manipulate instance of the virtualbinding to be displayed such that the virtual binding is to appear to bemanipulated in response to performance of the gesture.
 6. The computerdevice of claim 5, further comprising: context circuitry to determineone or more contextual attributes, wherein the one or more contextualattributes comprise user activity, semantic location, socialcircumstances, ambient conditions, presence of one or more electronicdevices, schedules, and user communications, wherein the AR circuitry isto generate the PO-manipulate instance based on the one or morecontextual attributes.
 7. The computer device of claim 1, wherein theinstance is a selection instance, and wherein: the AR circuitry is to:generate a virtual object (VO) when the gesture is a gesture to indicateto select the VO, generate a VO instance of the virtual binding when thegesture is a gesture to indicate to associate the virtual binding withthe VO, identify one or more virtual binding to VO (VB-VO) attributes ofthe association of the virtual binding with the VO, and generateVO-manipulate instance of the virtual binding when the gesture is agesture to manipulate the virtual binding; and the interface circuitryto: provide the VO and the VO instance of the virtual binding to bedisplayed, wherein the one or more VB-VO attributes are to indicatewhether, upon display of the VO instance and the VO, the virtual bindingis to appear to be attached to the VO or appear to interact with the VO,and provide the VO-manipulate instance of the virtual binding to bedisplayed such that the virtual binding is to appear to be manipulatedin response to performance of the gesture to manipulate the virtualbinding.
 8. The computer device of claim 1, wherein the modelingcircuitry is to generate a three-dimensional (3D) model of the physicalenvironment, and wherein the object recognition circuitry to identifythe physical object within the 3D model.
 9. The computer device of claim1, wherein the instance is a selection instance, and wherein: the ARcircuitry is to: generate a instrumented object (IO) instance of thevirtual binding when the gesture is a gesture to indicate to associatethe virtual binding with an instrumented object, and identify one ormore virtual binding to IO (VB-IO) attributes of the association of thevirtual binding with the VO; and the interface circuitry to: cause anetwork connection to be established with the instrumented object, andprovide the IO instance of the virtual binding to be displayed, whereinthe one or more VB-IO attributes are to indicate whether, upon displayof the IO instance, the virtual binding is to appear to be attached tothe instrumented object or appear to interact with the instrumentedobject.
 10. The computer device of claim 9, wherein: the AR circuitry isto generate a IO-manipulate instance of the virtual binding when thegesture is a gesture to manipulate the virtual binding to act on theinstrumented object; and the interface circuitry to: provide theIO-manipulate instance of the virtual binding to be displayed such thatthe virtual binding is to appear to be manipulated in response toperformance of the gesture, and cause a message to be transmitted to theinstrumented object over the network connection, wherein the instructionis to instruct the instrumented object to perform one or more actions inresponse to the manipulation of the virtual binding.
 11. One or morecomputer-readable media (CRM) comprising instructions, which whenexecuted by one or more processors of a computer device, is to cause thecomputer device to: obtain first sensor data from a sensor array,wherein the first sensor data is representative of a physicalenvironment; detect an object in the physical environment based on thefirst sensor data; identify the object and object attributes associatedwith the identified object; obtain, from the sensor array, second sensordata that is representative of a gesture performed by a user; identifythe gesture based on the second sensor data; generate, in response toidentification of the gesture, an instance of a virtual binding based onthe object attributes, wherein the instance of the virtual binding hasone or more virtual binding attributes that influence one or moreactions to be performed by the virtual binding; and provide the instanceof the virtual binding to be displayed such that, upon display of theinstance, the virtual binding is to appear to be attached to the objector appear to interact with the object.
 12. The one or more CRM of claim11, wherein the instance of the virtual binding is a selection instancewhen the gesture is a gesture to indicate a selection of a particularvirtual binding.
 13. The one or more CRM of claim 11, wherein executionof the instructions is to cause the computer device to obtain an objectprofile associated with the identified object, identify the objectattributes from the object profile, and determine the virtual bindingattributes based on the object attributes.
 14. The one or more CRM ofclaim 11, wherein the object is a physical object, and execution of theinstructions is to cause the computer device to: generate an associationinstance of the virtual binding when the gesture is a gesture toindicate to associate the virtual binding with the physical object;identify one or more virtual binding to physical object (VB-PO)attributes of the association of the virtual binding with the physicalobject; and provide the association instance of the virtual binding tobe displayed, wherein the one or more VB-PO attributes are to indicatewhether, upon display of the association instance, the virtual bindingis to appear to be attached to the physical object or appear to interactwith the physical object.
 15. The one or more CRM of claim 14, whereinexecution of the instructions is to cause the computer device to:generate a PO-manipulate instance of the virtual binding when thegesture is a gesture to manipulate the virtual binding; and provide thePO-manipulate instance of the virtual binding to be displayed such thatthe virtual binding is to appear to be manipulated in response toperformance of the gesture and according to the virtual bindingattributes.
 16. A method to be performed by a computer device, themethod comprising: obtaining, by the computer device, first sensor datafrom a sensor array, wherein the first sensor data is representative ofa physical environment; generating, by the computer device, athree-dimensional (3D) model of the physical environment for an ARenvironment based on the sensor data; identifying, by the computerdevice, an object within the 3D model and object attributes associatedwith the identified object; obtain, by the computer device from thesensor array, second sensor data that is representative of a gestureperformed by a user; identifying, by the computer device, the gesturebased on the second sensor data; generating, by the computer device inresponse to identification of the gesture, an instance of a virtualbinding based on the object attributes, wherein the instance of thevirtual binding has one or more virtual binding attributes thatinfluence one or more actions to be performed by the virtual binding;and providing, by the computer device, the instance of the virtualbinding to be displayed such that, upon display of the instance, thevirtual binding is to appear to be attached to the object or appear tointeract with the object.
 17. The method of claim 16, furthercomprising: generating, by the computer device, a virtual object (VO)when the gesture is a gesture to indicate to select the VO; generating,by the computer device, a VO instance of the virtual binding when thegesture is a gesture to indicate to associate the virtual binding withthe VO; identifying, by the computer device, one or more virtual bindingto VO (VB-VO) attributes of the association of the virtual binding withthe VO; and providing, by the computer device, the VO and the VOinstance of the virtual binding to be displayed, wherein the one or moreVB-VO attributes are to indicate whether, upon display of the VOinstance and the VO, the virtual binding is to appear to be attached tothe VO or appear to interact with the VO and in accordance with thevirtual binding attributes.
 18. The method of claim 17, whereinexecution of the instructions is to cause the computer device to:generating, by the computer device, VO-manipulate instance of thevirtual binding when the gesture is a gesture to manipulate the virtualbinding; and providing, by the computer device, the VO-manipulateinstance of the virtual binding to be displayed such that the virtualbinding is to appear to be manipulated in response to performance of thegesture and according to the virtual binding attributes.
 19. The methodof claim 16, wherein the instance is a first instance, and whereinexecution of the instructions is to cause the computer device to:generating, by the computer device, a instrumented object (IO) instanceof the virtual binding when the gesture is a gesture to indicate toassociate the virtual binding with an instrumented object; identifying,by the computer device, one or more virtual binding to IO (VB-IO)attributes of the association of the virtual binding with the VO;establishing, by the computer device, a network connection with theinstrumented object; and providing, by the computer device, the IOinstance of the virtual binding to be displayed, wherein the one or moreVB-IO attributes are to indicate whether, upon display of the IOinstance, the virtual binding is to appear to be attached to theinstrumented object or appear to interact with the instrumented objectand according to the virtual binding attributes.
 20. The method of claim19, wherein execution of the instructions is to cause the computerdevice to: generating, by the computer device, a IO-manipulate instanceof the virtual binding when the gesture is a gesture to manipulate thevirtual binding to act on the instrumented object; providing, by thecomputer device, the IO-manipulate instance of the virtual binding to bedisplayed such that the virtual binding is to appear to be manipulatedin response to performance of the gesture; and transmitting, by thecomputer device, a message to the instrumented object over the networkconnection, wherein the instruction is to instruct the instrumentedobject to perform one or more actions in response to the manipulation ofthe virtual binding.
 21. An augmented reality (AR) system, comprising:an environment comprising a sensor array and an output device array,wherein the sensor array comprises one or more sensors and the outputdevice array comprises one or more electromechanical devices; and an ARplatform communicatively coupled with the sensor array and the outputdevice array, the AR platform comprising: processor circuitry coupledwith memory circuitry, communication circuitry, and interface circuitry,wherein the processor circuitry is to: control the interface circuitryor the communication circuitry to obtain first sensor data from the oneor more sensors of the sensor array; generate a three-dimensional (3D)model of the environment based on the first sensor data; identify anobject within the 3D model and object attributes associated with theidentified object; obtain second sensor data from the one or moresensors of the sensor array, wherein the second sensor data isrepresentative of a gesture performed by a user; identify the gesturebased on the second sensor data; generate, in response to identificationof the gesture, an instance of a virtual binding based on the objectattributes, wherein the instance of the virtual binding has one or morevirtual binding attributes that influence one or more actions to beperformed by the virtual binding; and control the interface circuitry orthe communication circuitry to provide the instance of the virtualbinding to an individual output device of the one or more output devicesof the output device array, wherein the individual output device is todisplayed the instance of the virtual binding such that, upon display ofthe instance, the virtual binding is to appear to be attached to theobject or appear to interact with the object.
 22. The AR system of claim21, wherein the processor circuitry is to: generate an associationinstance of the virtual binding when the gesture is a gesture toindicate to associate the virtual binding with the physical object;identify one or more virtual binding to physical object (VB-PO)attributes of the association of the virtual binding with the physicalobject; and control the interface circuitry or the communicationcircuitry to provide the association instance of the virtual binding tobe displayed, wherein the one or more VB-PO attributes are to indicatewhether, upon display of the association instance, the virtual bindingis to appear to be attached to the physical object or appear to interactwith the physical object; generate a PO-manipulate instance of thevirtual binding when the gesture is a gesture to manipulate the virtualbinding; and control the interface circuitry or the communicationcircuitry to provide the PO-manipulate instance of the virtual bindingto be displayed such that the virtual binding is to appear to bemanipulated in response to performance of the gesture and according tothe virtual binding attributes.
 23. The AR system of claim 21, theprocessor circuitry is to: generate VO-manipulate instance of thevirtual binding when the gesture is a gesture to manipulate the virtualbinding; and control the interface circuitry or the communicationcircuitry to provide the VO-manipulate instance of the virtual bindingto be displayed such that the virtual binding is to appear to bemanipulated in response to performance of the gesture and according tothe virtual binding attributes.
 24. The AR system of claim 21, theprocessor circuitry is to: generate a instrumented object (IO) instanceof the virtual binding when the gesture is a gesture to indicate toassociate the virtual binding with an instrumented object; identify oneor more virtual binding to IO (VB-IO) attributes of the association ofthe virtual binding with the VO; cause a network connection to beestablished with the instrumented object; and control the interfacecircuitry or the communication circuitry to provide the IO instance ofthe virtual binding to be displayed, wherein the one or more VB-IOattributes are to indicate whether, upon display of the IO instance, thevirtual binding is to appear to be attached to the instrumented objector appear to interact with the instrumented object and according to thevirtual binding attributes.
 25. The AR system of claim 24, the processorcircuitry is to: generate a IO-manipulate instance of the virtualbinding when the gesture is a gesture to manipulate the virtual bindingto act on the instrumented object; control the interface circuitry orthe communication circuitry to provide the IO-manipulate instance of thevirtual binding to be displayed such that the virtual binding is toappear to be manipulated in response to performance of the gesture; andcontrol the communication circuitry to transmit a message to theinstrumented object over the network connection, wherein the instructionis to instruct the instrumented object to perform one or more actions inresponse to the manipulation of the virtual binding.